Year: 2012

  • procps-ng 3.3.5

    Are you using procps 3.3.4?  My suggestion is you don’t!

    An API change slipped through which mean if you tried to run procps 3.3.5 on libproc 3.3.4 it did strange things. One of those strange things is ps ax crashes. Top looked rather interesting as well.  This in turn upsets the boot sequence around mounting drives so you get a horrible mess.  The problem is the testsuite didn’t catch it because we’re not testing the API yet and the binaries use the libtool hack to use the library in the build directory.  Lucky for me I only updated manually the binaries and not the library so I saw it early.

    procps 3.3.5 was released tonight and it has the SONAME bump ,so we are not at libprocps1 not libprocps0.

    Enhanced by Zemanta
  • More spam from nobistech.net

    I get a lot of spam.  Most of it, thankfully is blocked by dspam but occasionally i get some through the filter.  One that particularly caught my eye was interesting not so much what it was advertising (I don’t read that part of the email) but where it came from and goes to.

    Normally there are two service providers involved in spam.  The email comes from (or via) one and then the spamvertised website is another.  The interesting thing is for this spam both of these were the same service provider. The email came from 174.34.168.85 and the spamvertised website was 70.32.40.194. Both of these addresses are owned by nobistech.net.  I punted the email to spamcop and it said that they’re not interested in spam reports.

    A few google queries shows that these guys seem quite happy to have spam sources and destinations and have been doing it for years.  They either appear as nobistech.net or unbiquity servers but they are one and the same organisation, or at least related.

    I won’t bother to send anything to them, it seems this has been done many many times by others with no results. Instead some CIDR blocks will be put into my blacklist.

    Enhanced by Zemanta
  • jqGridWidget in Turbogears

     

     

    Turbogears 2 uses Toscawidgets 2 for a series of very clever widgets and base objects that you can use in your projects.  One I have been playing with is the jqGridWidget which uses jquery to display a grid. The idea with jquery is creating a grid or other object and then using javascript to call back to the webserver to obtain the data which usually comes from a database using the very excellent SQLAlchemy framework.

    I found it a bit fiddly to get going and it is unforgiving, you get one thing wrong and you end up with an empty grid but when it works, it works very well.

    from tw2.jqplugins.jqgrid import jqGridWidget
    from tw2.jqplugins.jqgrid.base import word_wrap_css
    
    class ItemsWidget(jqGridWidget):
        def prepare(self):
            self.resources.append(word_wrap_css)
            super(ItemsWidget, self).prepare()
        options = {
                'pager': 'item-list-pager2',
                'url': '/items/jqgrid',
                'datatype': 'json',
                'colNames': ['Date', 'Type', 'Area', 'Description'],

    The code above is a snippet of the widget I use to make the graph. This is the visual side of the graph and shows just the  graph with nothing in it.  You need the datatype and url lines in the options which tell the code the url to get the data.  I have an items controller and within it there is a jqgrid method.  Note the number of columns I have, the data sent by the jqgrid method has to match this.

    Deep inside the items controller is a jqgrid method. The idea is to return some rows of data back to the grid in the exact way the grid expects it otherwise you get, yep more empty grids.

       @expose('json')
        def jqgrid(self, page=1, rows=30, sidx=1, soid='asc', _search='false',
                searchOper=u'', searchField=u'', searchString=u'', **kw):
    
            qry = DBSession.query(Item)
            qry = qry.filter()
            qry = qry.order_by()
            result_count = qry.count()
            rows = int(rows)
    
            offset = (int(page)-1) * rows
            qry = qry.offset(offset).limit(rows)
    
            records = [{'id': rw.id,
               'cell': [ rw.created.strftime('%d %b %H:%M:%S'), rw.item_type.display_name), rw.display_name, rw.text()]} for rw in qry]
            total = int(math.ceil(result_count / float(rows)))
            return dict(page=int(page), total=total, records=result_count, rows=records)

    The four items in the value for the cell in the dictionary must equal the number defined for the colNames.  I found it was useful to have firebug looking at my gets and another widget based upon SQLAjqGridWidget which is a lot simpler to setup but much less flexible too.

     

    Enhanced by Zemanta
  • NTP control messages

    I have had an opportunity to rework some code to query NTP servers directly in python rather than running ntpq -pn and scraping the result.  The over-the-wire protocol format is reasonably straight forward and my little module is now passing all of its nosetests which is wonderful.

    While its only a single table, ntpq actually does multiple requests to get its results.  NTP control messages are all mode 6 which tells the client and server its not time sort of things, but more about queries. A control packet has a sequence number which the server responds with so you can line up your requests with responses.  For large responses, the more, offset and count fields work together to knit up a large block of data.  The first packet of the response will have the offset field of 0. The count is the size of the data payload in bytes.  If the response is too big, the more bit will be set to 1 to say there is more coming.  The second packet will have an offset that is the same value as the previous packets count value, so you can stitch them together.

    The first query is a READSTAT which uses opcode 1.  This returns a list of 4 byte responses with the association ID taking the first two bytes and the peer status taking the second.  If you look at the output of ntpq you will notice a character in the first column which is the peer selection.  The important one for us is asterix, which equates to 110 in the lower 3 bits of the first status byte.  This is the system peer which is the main one. There may be others, up to 2, (usually symbol +) which are compared to the system peer.  ntpq doesn’t really care about the different peers and just displays the status but I want to know what peer the server is synchronised to.

    Next we need more details about the system peer (or all peers in ntpq).  To do this a READVAR or opcode 2 is sent to the server. The association ID is set to the association or peer we are interested in.  The response for my setup is sent over two packets which need to be reassembled and it is all in plain ascii which is a comma-separated list with a key=value pair.  I use the following python line to split it up.

    self.assoc_data.update(dict([tuple(ad.strip().split("=")) for ad in data[header_len:].split(",") if ad.find("=") > -1]))

    I’m sure people that know python better than I will say I am doing it wrong (it looks a bit klunky to me too) but it does work. From that I can query the various values that are returned by the server about its peers, including the source of its clock.

    Enhanced by Zemanta
  • Anti-Spam Fails

    A day or two ago I tried sending an email to a friend who happens to use the Road Runner ISP for his mail service.  Now this ISP doesn’t like dynamic IP addresses (using the increasing inaccurately named Dial Up List) so I have to punt the email through my ISP’s mailserver first.  Now that server is telling me this:

    The reason for the problem: 4.3.2 – Not accepting messages at this time 554-‘5.7.1 – ERROR: Mail refused – <150.101.137.131> – See http://sendersupport.senderscore.net’

    So their ISP mailserver is refusing connections from my ISP’s mailserver for some reason, probably on some spam list.  There’s a URL to look up the problem, so going there gives you three things:

    1. A redirection to https://sendersupport.senderscore.net/
    2. A badly configured webserver that uses the above URL with a certificate for www.senderscore.net
    3. A page that says “It works”

    Either Road Runner or Senderscore, preferably both, need to get a clue. Oh and going to https://www.senderscore.net/ gives connection refused. A bit of digging around shows the correct URL is https://senderscore.org/

    Now I just realized that my SSL certificate for https://enc.com.au/ expired on the weekend so I know these things can happen, but I’m one person (who was away for a while), why can’t companies get their act together?

    Enhanced by Zemanta
  • Careful with PIDs

    Quick question, what is the lowest Process ID you will find?  Most people (myself included until recently) would be able to say that the lowest number is 1 for the init process.  Plenty of programs including ps and pstree have this assumption.

    This assumption bit me this week with Debian Bug 687829 where pstree on a kFreeBSD would not return and used 100% CPU.  What was happening? Well it seems that kFreeBSD has a process ID of 0 and pstree used 0 for a fake root to the tree.  So with a real and fake process #0 hanging around in pstree there was a strange parent-child circle (I’m my own grandad!) between init and process 0 meaning when the tree was walked things go around in circles.

    The fix is basically to assume the tree starts at 0 and not 1 and that keeps things into perspective. So if you are wondering how inconsistent this goes, here is the list.

     

    • Linux has init  with PID 1 and PPID  0. There is no process 0
    • FreeBSD has init with PID 1 and PPID 0. Process 0 is kernel and exists and has PPID 0
    • Hurd has init at 1 with PPID 1. Process 0 is proc and has PPID 1

    So Linux the root is init PID 1, FreeBSD it is kernel PID 0 and Hurd its init PID 1 with a child with PID 0.

     

    Nothing like consistency! I do like the crazyness of Hurd where a child is created before a parent; nothing is normal in Hurd-land.

    I have some temporary code going proving the problem with psmisc and FreeBSD, the tree now looks like what you can see below.  This fix will also show up processes “hidden” under init that were previously hiding.

    ?()-+-acpi_cooling0(16)
        |-acpi_thermal(15)
        |-audit(10)
        |-bufdaemon(18)
        |-flowcleaner(22)
        |-g_down(4)
        |-g_event(2)
        |-g_up(3)
        |-hald-runner(547)---hald-addon-storag(569)
        |-hald(546)-+-hald(548)---hald(549)
        |           `-hald-runner(547)---hald-addon-storag(569)
        |-hald(548)---hald(549)
        |-idle(11)
        |-init(1)-+-etc....

     

    Enhanced by Zemanta
  • Australia Data Retention Scheme Stalled

    It’s been reported in the mainstream media that the Australian Data Retention Scheme has been referred to another committee, effectively delaying it to after the next election.  The Scheme was part of a broad review of the various security agencies powers and how to streamline them.  The initial paper originally stated that retention would be for up to 2 years for parts of the data set, without really specifying what the data was.

    There has been various ideas of what this data is, depending who you ask and when.  Some documents state it is only the accounting data; that user X used IP address Y at time Z, while others state it is email logs and a third have been the previous two plus web logs.

    There are many problems with a scheme such as this.  2 years of everyones web-browsing history is a desirable target for both legitimate and illegitimate access to that data.  Leaving aside would the access to the agencies get the right level of access; who else would want this data?  I’m sure that AFACT (Australia’s MPAA effectively) would like this data to go trawling.  The recent exposure of AAPT‘s data by Anonymous shows that there may be some other means of obtaining this data.

    I actually wonder, if the scheme was in, what they would find?  Take email for example; my email leaves my computer and directly communicates with the destination mail server. If someone pulled my email records they’d show very little information.  If it is too hard to run your own mailserver, I’m sure the enterprising no-gooders that they are so interested in finding can work out how to buy a remote mailserver somewhere.

    If that’s too hard, there’s always gmail with https, or VPNs, or TOR, or.. well there are plenty of options.  It probably comes down to how resourceful (if thats the word) people are.  By the way, mentioning https reminded me of the great little plug-in called HTTPS Everywhere by EFF.

    I’m glad to see that this legislation is stalled. To me the paperwork I saw appeared to be saying that they got some bad guys with what powers they have so they will catch more with more powers; therefore more powers are good, OK?  It’s a rather simplistic view of the world.  My only worry is its not stopped, just stalled so it might be re-animated in future.

     

    Enhanced by Zemanta
  • Context sensitive menus with GtkTreeView

    A project I now maintain called GJay uses the GTK+ toolkit and has a GtkTreeView that shows the directories and the songs in them.  I was wanting to add a context sensitive menu on that view so that you could do things to the item you have right-clicked on; generally file specific things like give me information about this song or queue it to play.  My only problem was, how to do it?  A few Google searches showed things that were almost, but not quite, what I was after.  The main problem being either the documentation was incomplete or it assumed I was after the selected field (the row I may of left-clicked on) and not the one I right-clicked on to get this event.

    This GtkTreeView uses a store where NAME_COLUMN is defined at 0, meaning it is the first column. The program uses this column to store the filename of the song.

    So first I needed to create an event to kick things off, we need to connect the “button-press-event” to our tree_view, which calls a function in this case view_on_button_pressed.

     g_signal_connect(tree_view, "button-press-event",
       G_CALLBACK(view_on_button_pressed), NULL);
    gboolean view_on_button_pressed(GtkWidget *tree_view,
                                    GdkEventButton *event,
                                    gpointer user_data) {
      GtkTreeSelection *selection;
      GtkTreePath *tree_path;
      GtkTreeModel *tree_model;
      GtkTreeIter tree_iter;
      gchar *name;
    
      if (event->type == GDK_BUTTON_PRESS && event->button == 3) {
        tree_model = gtk_tree_view_get_model(GTK_TREE_VIEW(tree_view));
        if (gtk_tree_view_get_path_at_pos(GTK_TREE_VIEW(tree_view),
              (int)event->x, (int)event->y,
              &tree_path, NULL, NULL, NULL)) {
          if (gtk_tree_model_get_iter(tree_model, &tree_iter, tree_path)) {
            gtk_tree_model_get(tree_model, &tree_iter, NAME_COLUMN, &name, -1);
            g_print("rc name is %sn", name);
          }
        }
      }

    So a breakdown of what my callback is doing:

    • Line 1:The callback’s widget is the GtkTreeView widget we connected the signal with
    • Line 10:Checks the event was a button press and button #3 (right button)
    • Line 11: Find the GtkTreeModel from the given GtkTreeView
    • Lines 12-14: Get the GtkTreePath that the right-click was sitting at by using the event coordinates.
    • Line 15: Convert the GtkTreePath into a GtkTreeIter
    • Line 16: Find the actual value we put into this row.

    The program, now that it knows the song title, can then go off and generate a menu for that song.

    Enhanced by Zemanta
  • debhelper versions – Guilty!

    After reading Jakub’s pet peeve about debhelper build-dependencies I decided to check my own and sponsored packages to see how they fare.

    find debian/ -type f -name control | xargs grep -h -o 'debhelper (>= 9[^)]*)' | sort | uniq -c
          2 debhelper (>= 9)
          2 debhelper (>= 9.0)
          3 debhelper (>= 9.0.0)
          1 debhelper (>= 9.20120115)

    Oops! Something to fix in all subsequent releases.

    Enhanced by Zemanta
  • Dejagnu tips

    Both the procps and psmisc projects use Dejagnu for their testing.  It’s interesting to try to understand how it all works and learning a new language TCL.  One of the big problems with Dejagnu is its documentation which is very sparse. So here are two things I’ve picked up along the way for a reference for myself and others.

    Locale

    We recently had a bug report on the procps list where tests for w failed for someone when the headers were shown.  After a some further debugging it was a locale problem, where the testsuite was looking for a number like 3.14 but the tester was seeing 3,14. There are many ways around this but for us it was to set the locale, with the following line in config/unix.exp after suggestions by Mike Frysinger should fix it.

    set env(LC_ALL) "C"

    Cleaning Up
    The documentation on this function is awesome! A title “Cleanup Procedure” and one line “cleanup()”.
    That’s all you get. The cleanup proc or function is called at the end of the testing. I use it
    to remove temporary files and things like that.

    proc cleanup { } {
            global test_file
            exec rm $test_file
    }