Category: Software

  • procps 3.3.6 and Mudlet 2.0

    Mudlet - Graphical MUD clientYesterday was a busy Debian day for me with the release of not one but two packages.

    procps

    procps version 3.3.6 was released both as an upstream and a Debian package.  While there were many bug fixes in this release, the main new feature for it was the top inspect feature.

    The top inspect feature allows you to run top in the normal way then you can inspect a specific process you have selected within top for more information.  Examples of this include fuser or lsof type output to see what files or libraries are open but really the limit is how clever the operator is and how often you would need this information.  Even scripts can be run off this feature if you have some customised or cooked output you need to see.  Once the output is there, you can search for text.

    There will be a small hold-up with the Debian packages as the section for the  library has been moved to libs, where it should be. At least I think that’s why there is a delay.

    procps infrastructure

    procps currently has only an email list and a gitorious git repository.  While this works most of the time, there are two main problems with this setup.

    The first is that the only two ways you can report bugs is send an email to the list or create a git merge request. The email has the problem with tracking and gathering history while the git sets the bar pretty high for reporting something has gone wrong.

    Second problem is there is no tarball file repository.  gitorious does have the feature of downloading tarballs of the tag checkpoints, but this is the raw repository.  When you make a tarball with “make dist” there are some generated files included in the tarball that don’t appear in the gitorious ones, which means its not a simple for some as you expect.

    So I’ll be creating a project for these two features soon.  Not sure where they will be but I use sourceforge for both these for a few other projects so that might be where they end up.

     

    Mudlet

    Mudlet, the multi-user dungeon or MUD client, has finally made it to version 2.0!  It looks a lot more stable and polished at least from the scripting side than the previous RC versions but has all the features that we’ve come to expect with it.  The most important thing is that the developers have finally put a line in the sand and said “this is version 2.0”.  The Debian package was uploaded yesterday and should be on its way to your local mirror soon.

     

    Enhanced by Zemanta
  • procps-ng 3.3.5

    Are you using procps 3.3.4?  My suggestion is you don’t!

    An API change slipped through which mean if you tried to run procps 3.3.5 on libproc 3.3.4 it did strange things. One of those strange things is ps ax crashes. Top looked rather interesting as well.  This in turn upsets the boot sequence around mounting drives so you get a horrible mess.  The problem is the testsuite didn’t catch it because we’re not testing the API yet and the binaries use the libtool hack to use the library in the build directory.  Lucky for me I only updated manually the binaries and not the library so I saw it early.

    procps 3.3.5 was released tonight and it has the SONAME bump ,so we are not at libprocps1 not libprocps0.

    Enhanced by Zemanta
  • jqGridWidget in Turbogears

     

     

    Turbogears 2 uses Toscawidgets 2 for a series of very clever widgets and base objects that you can use in your projects.  One I have been playing with is the jqGridWidget which uses jquery to display a grid. The idea with jquery is creating a grid or other object and then using javascript to call back to the webserver to obtain the data which usually comes from a database using the very excellent SQLAlchemy framework.

    I found it a bit fiddly to get going and it is unforgiving, you get one thing wrong and you end up with an empty grid but when it works, it works very well.

    from tw2.jqplugins.jqgrid import jqGridWidget
    from tw2.jqplugins.jqgrid.base import word_wrap_css
    
    class ItemsWidget(jqGridWidget):
        def prepare(self):
            self.resources.append(word_wrap_css)
            super(ItemsWidget, self).prepare()
        options = {
                'pager': 'item-list-pager2',
                'url': '/items/jqgrid',
                'datatype': 'json',
                'colNames': ['Date', 'Type', 'Area', 'Description'],

    The code above is a snippet of the widget I use to make the graph. This is the visual side of the graph and shows just the  graph with nothing in it.  You need the datatype and url lines in the options which tell the code the url to get the data.  I have an items controller and within it there is a jqgrid method.  Note the number of columns I have, the data sent by the jqgrid method has to match this.

    Deep inside the items controller is a jqgrid method. The idea is to return some rows of data back to the grid in the exact way the grid expects it otherwise you get, yep more empty grids.

       @expose('json')
        def jqgrid(self, page=1, rows=30, sidx=1, soid='asc', _search='false',
                searchOper=u'', searchField=u'', searchString=u'', **kw):
    
            qry = DBSession.query(Item)
            qry = qry.filter()
            qry = qry.order_by()
            result_count = qry.count()
            rows = int(rows)
    
            offset = (int(page)-1) * rows
            qry = qry.offset(offset).limit(rows)
    
            records = [{'id': rw.id,
               'cell': [ rw.created.strftime('%d %b %H:%M:%S'), rw.item_type.display_name), rw.display_name, rw.text()]} for rw in qry]
            total = int(math.ceil(result_count / float(rows)))
            return dict(page=int(page), total=total, records=result_count, rows=records)

    The four items in the value for the cell in the dictionary must equal the number defined for the colNames.  I found it was useful to have firebug looking at my gets and another widget based upon SQLAjqGridWidget which is a lot simpler to setup but much less flexible too.

     

    Enhanced by Zemanta
  • NTP control messages

    I have had an opportunity to rework some code to query NTP servers directly in python rather than running ntpq -pn and scraping the result.  The over-the-wire protocol format is reasonably straight forward and my little module is now passing all of its nosetests which is wonderful.

    While its only a single table, ntpq actually does multiple requests to get its results.  NTP control messages are all mode 6 which tells the client and server its not time sort of things, but more about queries. A control packet has a sequence number which the server responds with so you can line up your requests with responses.  For large responses, the more, offset and count fields work together to knit up a large block of data.  The first packet of the response will have the offset field of 0. The count is the size of the data payload in bytes.  If the response is too big, the more bit will be set to 1 to say there is more coming.  The second packet will have an offset that is the same value as the previous packets count value, so you can stitch them together.

    The first query is a READSTAT which uses opcode 1.  This returns a list of 4 byte responses with the association ID taking the first two bytes and the peer status taking the second.  If you look at the output of ntpq you will notice a character in the first column which is the peer selection.  The important one for us is asterix, which equates to 110 in the lower 3 bits of the first status byte.  This is the system peer which is the main one. There may be others, up to 2, (usually symbol +) which are compared to the system peer.  ntpq doesn’t really care about the different peers and just displays the status but I want to know what peer the server is synchronised to.

    Next we need more details about the system peer (or all peers in ntpq).  To do this a READVAR or opcode 2 is sent to the server. The association ID is set to the association or peer we are interested in.  The response for my setup is sent over two packets which need to be reassembled and it is all in plain ascii which is a comma-separated list with a key=value pair.  I use the following python line to split it up.

    self.assoc_data.update(dict([tuple(ad.strip().split("=")) for ad in data[header_len:].split(",") if ad.find("=") > -1]))

    I’m sure people that know python better than I will say I am doing it wrong (it looks a bit klunky to me too) but it does work. From that I can query the various values that are returned by the server about its peers, including the source of its clock.

    Enhanced by Zemanta
  • Careful with PIDs

    Quick question, what is the lowest Process ID you will find?  Most people (myself included until recently) would be able to say that the lowest number is 1 for the init process.  Plenty of programs including ps and pstree have this assumption.

    This assumption bit me this week with Debian Bug 687829 where pstree on a kFreeBSD would not return and used 100% CPU.  What was happening? Well it seems that kFreeBSD has a process ID of 0 and pstree used 0 for a fake root to the tree.  So with a real and fake process #0 hanging around in pstree there was a strange parent-child circle (I’m my own grandad!) between init and process 0 meaning when the tree was walked things go around in circles.

    The fix is basically to assume the tree starts at 0 and not 1 and that keeps things into perspective. So if you are wondering how inconsistent this goes, here is the list.

     

    • Linux has init  with PID 1 and PPID  0. There is no process 0
    • FreeBSD has init with PID 1 and PPID 0. Process 0 is kernel and exists and has PPID 0
    • Hurd has init at 1 with PPID 1. Process 0 is proc and has PPID 1

    So Linux the root is init PID 1, FreeBSD it is kernel PID 0 and Hurd its init PID 1 with a child with PID 0.

     

    Nothing like consistency! I do like the crazyness of Hurd where a child is created before a parent; nothing is normal in Hurd-land.

    I have some temporary code going proving the problem with psmisc and FreeBSD, the tree now looks like what you can see below.  This fix will also show up processes “hidden” under init that were previously hiding.

    ?()-+-acpi_cooling0(16)
        |-acpi_thermal(15)
        |-audit(10)
        |-bufdaemon(18)
        |-flowcleaner(22)
        |-g_down(4)
        |-g_event(2)
        |-g_up(3)
        |-hald-runner(547)---hald-addon-storag(569)
        |-hald(546)-+-hald(548)---hald(549)
        |           `-hald-runner(547)---hald-addon-storag(569)
        |-hald(548)---hald(549)
        |-idle(11)
        |-init(1)-+-etc....

     

    Enhanced by Zemanta
  • Context sensitive menus with GtkTreeView

    A project I now maintain called GJay uses the GTK+ toolkit and has a GtkTreeView that shows the directories and the songs in them.  I was wanting to add a context sensitive menu on that view so that you could do things to the item you have right-clicked on; generally file specific things like give me information about this song or queue it to play.  My only problem was, how to do it?  A few Google searches showed things that were almost, but not quite, what I was after.  The main problem being either the documentation was incomplete or it assumed I was after the selected field (the row I may of left-clicked on) and not the one I right-clicked on to get this event.

    This GtkTreeView uses a store where NAME_COLUMN is defined at 0, meaning it is the first column. The program uses this column to store the filename of the song.

    So first I needed to create an event to kick things off, we need to connect the “button-press-event” to our tree_view, which calls a function in this case view_on_button_pressed.

     g_signal_connect(tree_view, "button-press-event",
       G_CALLBACK(view_on_button_pressed), NULL);
    gboolean view_on_button_pressed(GtkWidget *tree_view,
                                    GdkEventButton *event,
                                    gpointer user_data) {
      GtkTreeSelection *selection;
      GtkTreePath *tree_path;
      GtkTreeModel *tree_model;
      GtkTreeIter tree_iter;
      gchar *name;
    
      if (event->type == GDK_BUTTON_PRESS && event->button == 3) {
        tree_model = gtk_tree_view_get_model(GTK_TREE_VIEW(tree_view));
        if (gtk_tree_view_get_path_at_pos(GTK_TREE_VIEW(tree_view),
              (int)event->x, (int)event->y,
              &tree_path, NULL, NULL, NULL)) {
          if (gtk_tree_model_get_iter(tree_model, &tree_iter, tree_path)) {
            gtk_tree_model_get(tree_model, &tree_iter, NAME_COLUMN, &name, -1);
            g_print("rc name is %sn", name);
          }
        }
      }

    So a breakdown of what my callback is doing:

    • Line 1:The callback’s widget is the GtkTreeView widget we connected the signal with
    • Line 10:Checks the event was a button press and button #3 (right button)
    • Line 11: Find the GtkTreeModel from the given GtkTreeView
    • Lines 12-14: Get the GtkTreePath that the right-click was sitting at by using the event coordinates.
    • Line 15: Convert the GtkTreePath into a GtkTreeIter
    • Line 16: Find the actual value we put into this row.

    The program, now that it knows the song title, can then go off and generate a menu for that song.

    Enhanced by Zemanta
  • debhelper versions – Guilty!

    After reading Jakub’s pet peeve about debhelper build-dependencies I decided to check my own and sponsored packages to see how they fare.

    find debian/ -type f -name control | xargs grep -h -o 'debhelper (>= 9[^)]*)' | sort | uniq -c
          2 debhelper (>= 9)
          2 debhelper (>= 9.0)
          3 debhelper (>= 9.0.0)
          1 debhelper (>= 9.20120115)

    Oops! Something to fix in all subsequent releases.

    Enhanced by Zemanta
  • Dejagnu tips

    Both the procps and psmisc projects use Dejagnu for their testing.  It’s interesting to try to understand how it all works and learning a new language TCL.  One of the big problems with Dejagnu is its documentation which is very sparse. So here are two things I’ve picked up along the way for a reference for myself and others.

    Locale

    We recently had a bug report on the procps list where tests for w failed for someone when the headers were shown.  After a some further debugging it was a locale problem, where the testsuite was looking for a number like 3.14 but the tester was seeing 3,14. There are many ways around this but for us it was to set the locale, with the following line in config/unix.exp after suggestions by Mike Frysinger should fix it.

    set env(LC_ALL) "C"

    Cleaning Up
    The documentation on this function is awesome! A title “Cleanup Procedure” and one line “cleanup()”.
    That’s all you get. The cleanup proc or function is called at the end of the testing. I use it
    to remove temporary files and things like that.

    proc cleanup { } {
            global test_file
            exec rm $test_file
    }
    
  • pam bugs hurt

    I did some upgrades of what seemed like a million packages today on my Debian sid computer.  I was doing this remotely and when I tried to ssh back in I got kicked out after entering my password, hmmm.

    OK, so I waited until I could get in front of it and tried to login to the console and was greeted with the message “Cannot make/remove an entry” and kicked out, even as root; double hmmm.

    So it was boot into single user time which, thankfully worked. bash loaded ok so it wasn’t a shell problem.  However su failed which means we are into pam problem territory.  I found a bug report that sounded like my problem, but with no clear solution except upgrade in Ubuntu.

     

    The solution was pretty simple for me.  It’s something funky with /etc/pam.d/common-sesssion and a simple ‘/usr/sbin/pam-auth-update’ fixed it for me.

     

     

    Enhanced by Zemanta
  • JFFNMS 0.9.3

    JFFNMS version 0.9.3 has been released today.  This is a vast improvement over the 0.9.x releases and anyone using that train is strongly recommended to upgrade.So what changed? What didn’t change!  A nice summary would be fixing a lot of things that were broken or needed some tweaking. A really, really big thanks to Marek for all the testing and bug reports and also patient “just run this and tell me what it says” tests he did too.  If something wasn’t right before and works now, it is quite likely it is working because Marek told me how it broke.

    A brief overview of what has changed:

    • TFTP transfers work again
    • A lot of the wierd polling effects due to caching fixed
    • Lots of the selects in sub-tables now work
    • The PHP string-to-float brokeness in SLAs worked-around
    • Even more SNMP library cruft removed or escaped
    • HostMIB apps match properly
    • Interface autodiscovery delete and update fields back working

    You can download the file off sourceforge at

    https://sourceforge.net/projects/jffnms/files/JFFNMS%20Releases/

    Enhanced by Zemanta