Mixing pysnmp and stdin

Depending on the application, sometimes you want to have some socket operations going (such as loading a website) and have stdin being read. There are plenty of examples for this in python which usually boil down to making stdin behave like a socket and mixing it into the list of sockets select() cares about.

A while ago I asked an email list could I have pysnmp use a different socket map so I could add my own sockets in (UDP, TCP and a zmq to name a few) and the Ilya the author of pysnmp explained how pysnmp can use a foreign socket map.
Read more Mixing pysnmp and stdin

Rnms now using TurboGears 2.3.2

While the website might of been idle there has been a great deal of work behind the scenes (unless you’re looking at github) updating and improving RoseNMS, the python based Network Management System. The code has had a great many changes including:

  • Updating the TurboGears based code to 2.3.2
  • Simplifying the admin back to TG standard using the Twitter Bootstrap option
  • Making several GUI changes to make it easier to switch screens or get more information
  • Update to the pysnmp module to the latest, which is a much better module
  • Changing from paste to cliff for the command lines and bringing it all into one binary

I’m hoping to get to release one in the next few months. The back-end is largely complete with some work required on the daemon and also more webGUI work to make sure its easy to get the right information at the right time.


A python utf gotcha

This one had me stumped for a while:

# -*- coding: utf-7 -*-
import datetime
from sqlalchemy import ForeignKey, Column
from sqlalchemy.types import Integer, Unicode, Boolean, DateTime

default_due_date = datetime.datetime.now() + datetime.timedelta(days=30)

Syntax error found on last line.

Hmm, bring up a python interpreter and type the last line in with the imports. Works fine.

It’s the first line that is the problem, I typoed it and made it utf-7 not utf-8. I suppose it means that it is case-insenstive. Still, it wasn’t too clear to me at least, what was going on.

Enhanced by Zemanta

Step or Sloping Graphs

Even though the backend of Rosenberg NMS uses rrdtool RRD files, the front end graphs are created using jqPlot.  The idea is to have a set of templates for the different types of graphs and just apply them to the various data sets.  It makes things a lot simpler for new graphs because you just select which one you want; unless you want something a lot different which would involve a new graph template.

In any case, anyone that looks enough at the standard rrdtool graphs will know they are a series of steps. While it depends on the RRA, usually they are 5 minute steps, so a graph showing an increasing rate might show 5 minutes of 2 Mbps and then the next 5 minutes of 11 Mbps. jqPlot graphs as I’ve currently got them draws a line between two data points, so there would be a sloping line starting at 2.5 minutes (half of the first 5 minute interval) and 2Mbps sloping up to 11 Mbps at 7.5 minutes.

At first I thought this was wrong and spent some time attempting to “fix” the graph by making it look like a rrdtool graph more. Someway through that process I stopped and wondered, what IS the right way? The answer like a lot of other things, is “it depends”.

For a graph showing a rate, such as the output bits per second on an interface, the way this is done is at regular intervals a counter is measured. So if at time 0 the counter is 140 and time 60 the counter is 200 and finally at time 120 the counter is at 800 there has been an average rate of 1 [(200-140)/60] and 100 [(800-200)/60)]. rrdtool would show a horizontal bar at 1 bps and then another horizontal bar at 100 at the next time interval. jqPlot would show a sloping line going from 30,1 up to 90,100.

Two graphs looking very different from the same data, what gives?  Each graph is right, or rather is showing an estimation of different things, hence their differences.

rrdtool is showing the average rate for that time period. It is in some ways accurate because leaving aside missed wrap-arounds and resets that many bits did pass through that interface for the time specified.  However often the graphs are interpreted incorrectly as the real rate and not an average.  We can be reasonably sure for a rate that it would not be 1 for exactly a minute and then immediately jump to 100 for another minute.  This isn’t rrdtool’s fault, it is just how the graphs can be interpreted.

jqplot will show more “realistic” graphs, with a curve sloping up. However this too makes assumptions that the rate increase is linear which often it is not. It just gives the illusion that the graph knows more about the data than it really does.

In the end, both graphs are at the same time accurate and misleading. It’s important when looking at any graph in general (not just these two types) that you understand its limits and assumptions.  To give one example of the problems that can be missed, traffic interfaces may have microbursts (large amount of traffic in short amount of time) which, due to the averaging that goes on in graphing are invisible to graphs and give an incorrect account of what is going on.



Enhanced by Zemanta

itools is back

My last post I said that I had to remove my internet query tools due to some bugs that were a concern.  Some of the code was hard to maintain and probably had holes and I had noticed that it looped at times.

I’m happy to say that I have restored some of those tools now, still located at http://enc.com.au/itools

This code is completely re-written in Python using the TurboGears toolkit which means it is a lot cleaner in how it works and how it looks.  Some of the lookup tables use a database rather than an array for ease of updating and querying.  The downside is the backends will take time.  It currently only does nslookup queries and whois only works for IPv4 addresses. The domain name queries will be a while off as these are the most complicated to handle. To give you an idea, all IPv4 and IPv6 address information comes from 5 sources with two formats while domain names come from over 200 sources with about 40 formats.  This means the information from Regional Internet Registries will be done first.

Enhanced by Zemanta