Author: dropbear

  • Odd WordPress pingbacks

    I’m getting some odd log messages for the apache module modsecurity.  Essentially its xml parser is breaking when random places are sending pingbacks. The requests go to xmlrpc.php and the response headers are ok, but the body is binary.  The message in the modsecurity log looks like:

    Message: XML parser error: XML: Failed parsing document.

    After a bit of guessing and sending messages to and fro, I can now see that it is a gziped response. So I’m not sure if it is modsecurity not realising that the response is gziped or wordpress not marking it correctly. In any case I can regularly get very similar binary strings using gzip and the usual xml response. So that’s half the mystery solved.

    Most of the requests are spammers so I’m not too worried.  I think it also impacts legitimate pingbacks because I’ve not had any, even from the usual automatic places.

  • pidof moving to procps

    pidof is a program that finds the PID of a named program. In some ways it is like a cut-down pgrep found in the procps package.  pidof currently sits in sysvinit-tools.

    There are plans to move all utilities that use the proc filesystem under one package which will make the maintenance of them simpler, which in effect means moving pidof from sysvinit-tools to procps. The short-term bump should make it better in the long term.

    Now as I wear two hats (Debian maintainer and procps upstream) there are two very important things I/we need to know.

    • If your Debian package depends on pidof being present, then we need to discuss dependencies. procps is generally installed on most systems but there might be corner cases. Possibly just depending on a specific version of procps will do it
    • If you, your Debian package or anything else (including other distributions) need the non-LSB options of pidof (ie they use -c -n or -m) then we (upstream) need to know about it. There are provisional plans not to support these options but they’re needed, or a subset is, then that could change.

    Debian developers can chime in on the debian-devel email list, or email myself. If its an upstream issue then either email me, or even better, the procps email list

    Enhanced by Zemanta
  • postfix transport with smarthost

    Dear lazyyweb,

    Has anyone got a way for postfix to use a transport map such that it sends email to the given MX host for some specified domains then default to a smarthost for the remainder?

    The logic would be:

    IF domain in (‘example.net’, ‘foo.bar’,etc etc) THEN use the relevant MX host ELSE send to smarthost.isp.net

    i can use a transport map to send from specific domains to specific hosts, eg @domain.com goes to otherisp.net mailserver but not to just use domain.com’s MX hosts

    Enhanced by Zemanta
  • Careful with apache upgrades

    You might (or not if you don’t visit) notice all my websites were down.  A rushed apt-get dist-upgrade and I found two problems:

    1. PHP5 got removed, which is bad if you run a wordpress site that uses PHP to run
    2. The apache configuration has changed.

    Yes, the NEWS entries did warn me, if I read them fully. Yes, I didn’t read them enough.

    Apache now ignores configuration files that don’t end in .conf To give a completely non-theoretical example, if you have your virtual hosts in files such as /etc/apache2/sites-enabled/enc.com.au then this will not be recognised and your sites will show the default “It works” page.

    Stuff that doesn’t fall in the usual places where website stuff should go, which for my setup is a lot of things, will also be denied as the developers have tightened up the rules around what is permitted.  Pretty simple to fix with a few <Directory blah> clauses.

    This isn’t a criticism of the Debian apache developers. They do an awesome job of keeping the package workable, flexible but secure which isn’t easy.  Now it’s all back working, I actually agree with the changes they have made. It is just that the latest changes are, well, tricky so be forewarned.

     

    Enhanced by Zemanta
  • Step or Sloping Graphs

    Even though the backend of Rosenberg NMS uses rrdtool RRD files, the front end graphs are created using jqPlot.  The idea is to have a set of templates for the different types of graphs and just apply them to the various data sets.  It makes things a lot simpler for new graphs because you just select which one you want; unless you want something a lot different which would involve a new graph template.

    In any case, anyone that looks enough at the standard rrdtool graphs will know they are a series of steps. While it depends on the RRA, usually they are 5 minute steps, so a graph showing an increasing rate might show 5 minutes of 2 Mbps and then the next 5 minutes of 11 Mbps. jqPlot graphs as I’ve currently got them draws a line between two data points, so there would be a sloping line starting at 2.5 minutes (half of the first 5 minute interval) and 2Mbps sloping up to 11 Mbps at 7.5 minutes.

    At first I thought this was wrong and spent some time attempting to “fix” the graph by making it look like a rrdtool graph more. Someway through that process I stopped and wondered, what IS the right way? The answer like a lot of other things, is “it depends”.

    For a graph showing a rate, such as the output bits per second on an interface, the way this is done is at regular intervals a counter is measured. So if at time 0 the counter is 140 and time 60 the counter is 200 and finally at time 120 the counter is at 800 there has been an average rate of 1 [(200-140)/60] and 100 [(800-200)/60)]. rrdtool would show a horizontal bar at 1 bps and then another horizontal bar at 100 at the next time interval. jqPlot would show a sloping line going from 30,1 up to 90,100.

    Two graphs looking very different from the same data, what gives?  Each graph is right, or rather is showing an estimation of different things, hence their differences.

    rrdtool is showing the average rate for that time period. It is in some ways accurate because leaving aside missed wrap-arounds and resets that many bits did pass through that interface for the time specified.  However often the graphs are interpreted incorrectly as the real rate and not an average.  We can be reasonably sure for a rate that it would not be 1 for exactly a minute and then immediately jump to 100 for another minute.  This isn’t rrdtool’s fault, it is just how the graphs can be interpreted.

    jqplot will show more “realistic” graphs, with a curve sloping up. However this too makes assumptions that the rate increase is linear which often it is not. It just gives the illusion that the graph knows more about the data than it really does.

    In the end, both graphs are at the same time accurate and misleading. It’s important when looking at any graph in general (not just these two types) that you understand its limits and assumptions.  To give one example of the problems that can be missed, traffic interfaces may have microbursts (large amount of traffic in short amount of time) which, due to the averaging that goes on in graphing are invisible to graphs and give an incorrect account of what is going on.

     

     

    Enhanced by Zemanta
  • jqplot in Turbogears

     

    I’ve been working on the Rosenberg jqPlot GraphsNMS graphs slowly migrating them from using rrdtool graph and using jqplot.  While there have been many false-starts and re-works, I now have a working set of graphs, two of which are shown on the page.

    The graphs look a lot slicker and I have also simplified the admin screens.  I found I kept having to type the same thing in over and over for the rrdgraphs and have narrowed down the type of graphs to approximately 5.  They’re certainly not bulletproof and need more testing but they’re a good start.

    The graphs are based upon the ToscaWidgets2 series of widgets that then provide a nice “handle” for the jqPlot code.  My graphs even have some hand-coded Javascript to give nice units on the Y axis.

     

    Enhanced by Zemanta
  • First Look at RoseNMS screens

    More work has gone into the various GUI screens for RoseNMS. While still early days, these screenshots give an idea of the final program.

    Host Map

    The maps have been updated and now just use CSS for layout, rather than hard calculating the coordinates using javascript. This gives a cleaner and more fluid look to the maps. Hovering over the map item will show more details about the item. There are maps for hosts (shown) as well as attributes.

    Attribute View with information, graph and events
    Attribute View with information, graph and events

    Attribute details show some information about the attribute, the default graph and the events. Clicking on the more button above the graph takes you to the graph page for that attribute.

    Graphs

    Finally the graph handling is completely re-written with the concept of graph templates. These are basic types of graphs. The one shown is the basic “lines” template that shows… lines! Only things required are the RRD data source, the multiplier, the legend and how you want the units shown. From that the program builds all the required VDEFs and CDEFs.

  • props-ng 3.3.8

    Procps-ng 3.38 was released yesterday.  Besides the bug fixes,there have been two main enhancements: NUMA and systemd support.

    top comes with NUMA support which is a soft dependency, meaning that if compilied with the support, top will try to link at runtime to libnuma. The new key presses are ‘2’ and ‘3’ which will show the Nodes or more detail about a specific node respectively.

    ps has two new output columns; unit and uunit. These permit ps to display the systemd unit and user unit fields.  The systemd support is a
    hard dependency which is enabled with –with-systemd

     

    procps-ng is available from gitorious or sourceforge:

    Enhanced by Zemanta
  • RNMS on GitHub

    Rosenberg NMS is now found on GitHub at https://github.com/csmall/rnms

    The code is still pre-alpha but it is now available to try out.  This means you can try out this new network management system. I have done this to hopefully shake out some of the bugs before the first release.  The program has minimal documentation but does work, kinda.

     

    Enhanced by Zemanta
  • Removing itools

    Update: The newer code is now running, though still needs some testing.

    For very many years I have been running a set of tools on my website that basically runs whois or nslookup queries and presents them in a standard format.  I have decided today to shut this part of the website down as the code running those components is very old and I’ve not maintained it for years.  Back when I initially wrote the tools, in 1995 or so, there wasn’t many alternatives to this site but that has long changed.

    So thanks for those who emailled me over the years; its been an interesting journey.

    Enhanced by Zemanta