Dropbear

Blog

  • Scary Functions: rawmemchr

    Recently I received a bug report for psmisc that basically supplied a patch so psmisc works in a cygwin environment.  Nothing terribly exciting in that and the changes are pretty uncontroversial.

    One of the changes is in that instead of using rawmemchr to find the end of the string, cygwin doesn’t have that function and so it is replaced by strchr.  I’ve not seen the function rawmemchr before so I had a closer look.

    Initially I was a little worried, rawmemchr tries to find the given byte starting at a given pointer but unlike strchr it doesn’t stop unless it finds the character. I think the only time this function is safe, or is no more evil than anything else, is when you are searching for the NUL character which is what fuser in psmisc does.

    Still, the string being checked is coming from getcwd which requires a string or buffer length so perhaps it would be better to use memchr which is a little slower but has a maximum length parameter.  If you have code that uses this function, you may just want to check how it is being used because it seems like an old bad gets function that plagued a lot of programs.

    Enhanced by Zemanta

  • First Look at Python

    When the programming language Python came out while it seemed like an interesting idea, I didn’t really see the point. I already know Perl, PHP and C (plus a few others) why learn another language? So until recently, I didn’t

    Recent frustrations with PHP and how “loose” it is with things like variables made me have another look at Python and what it could do for me.  As just learning a language by itself it pretty boring I decided to write some small programs in the language and used the framework Turbogears to do it.

    First of all, learning the basics of python from perl or PHP is dead simple. The idea of what a language looks like is not too different between these three so it was a snap to write simple stuff in python.  So, what does yet-another language give me?

    • Variables must be defined before being used, but assigning them defines them too.  This is a nice compromise between PHP where you can check undefined variables and C where it all must be defined first, or else.
    • try and except. For code that might fail, use try and then catch the error in except.  We’ve all been guilty of using the @ symbol in PHP to supress stupid error messages, this does it correctly.
    • No PHP extract() or compact(). Actually just no extract function is good enough. That function is pure evil.
    • Sqlalchemy – A proper “you look after the database and ill play with objects” database extraction layer thingamy.  I don’t want to play with databases just let me write code and thats what sqlalchemy does well.
    • Testing – Turbogears uses nosetests which does functional and unit testing. I do get some annoying artifacts based upon testing but generally its all good.
    • Python has help() everywhere and pydoc on the command line. Not sure what variable foo can do? Just type “help(foo)” or even stick it in your script.
    • paster! You can run a development site off the command-line using a simple sqlite database. Don’t like it, dump the DB and reinstall it with one command.

    It’s not one-way though, there have been some challenges.  Quite likely these are more my deficiencies than the languages.

    • Error messages take up a whole screen and often make it difficult to isolate. It’s not quite 10-screens-full ala Eiffel but sometimes the error messages tell you something is going wrong, but not what. Mess up your object versus database table references and you get something rather obscure.
    • When is an integer not an integer? The answer is, sometimes. Sometimes they work, sometimes they break. It’s probably more to do with what pysnmp is doing with them.
    • Documentation – some is great, some is downright misleading. It comes down to the writer of the documentation for modules. At least it doesn’t have ‘well get round to it’ like some functions in PHP on their website. Documents that say you can directly iterate dictionaries were another dead-end I wandered down. If you are stuck here, add a .items() at the end of it.
    • Deployment. It’s not just a matter of taring the files up and sticking them into /var/www/sitename and you’re done.  There are probably good reasons why, but its just annoying.

    Despite the minor problems, python really does give you much much more than PHP and some more than Perl.  For me and what I am intending on writing it is worth learning yet another language.

    Related articles
    Enhanced by Zemanta

  • psmisc 22.14 Released

    The upstream and Debian packages for psmisc version 22.14 were released tonight.  This version has a lot of bugfixes.  I’ve also taken some time to clean up some of the minor compile warnings I used to get which were mainly variables set but not used.

    Two bugs that I’ll concentrate on getting fixed for the next version are all about the command line.

    The first is that the fuser -s flag changes the return code for items found.  It should be consistent so it doesnt matter if fuser displays values or not, 0 means found something and 1 means didn’t find it. I can see why in the code it might be different, because the code follows different paths with the s option, but not exactly where the problem is.

    The second bug is the -M flag for fuser only works if it is before the mount points so -M -m /foo works but -m /foo -M does not.  Unlike the -s problem, I can see where it is failing.  Currently fuser scans the options one by one.  If it finds a -m option, it checks at that point if the -M option is set.  The solution is to store all potential targets and only after exhausting the entire command line do we add the potential targets  to our “real” targets.  It means scanning the command line, or a subset of it, twice but the performance penalty is tiny.

    Finally, I released there was a rather neat Makefile command “make distcheck”. This target makes a tar.g archive, extracts it, builds then cleans.  It is a very good program for trapping things like a file exists in your local directory but is not included in the tar.gz (lists.h was missing until I ran this).  For some “make distcheck” is something they’re known for years, but if you have an autotools package you develop, have a look at it.

    With all the talk about testing and test driven releases, I was going to put some test cases in. The problem is there doesn’t seem to be an easy way to do testing. I might have to just hack up some scripts but surely there is a better way. That would trap, for example that fuser -s returns 1 but fuser returns 0.  I looked at check, but it doesn’t work for single-file programs like psmisc is mainly made of.

    Enhanced by Zemanta

  • Debian Linux on HP DV6-6023TX

    It was time to update my creaking old laptop.  I ordered HP6-6023TX online and after a few days it arrived just before a long weekend.

    I first backed up Windows 7 onto a set of DVDs and found that either my blank DVDs are old or this new DVD drive is far more fussy. There were quite a few coasters made until I gave up and bought some new ones.

    DVDs seemed to be trouble for me that day.  I downloaded the Debian 6.0.4 DVD image but couldn’t burn it. Brasero would just keep saying calculating checksum and go no further.  So after waiting too long, the CD iso was downloaded and installed.

    Installation of Debian onto the laptop was straightforward. That is if you used a wired connection. Wireless needs the non-free firmware and I gave up trying to work out how to install it at debian-installer time as it didn’t see the tar.gz full of firmware on the USB stick.  Wireless worked perfectly once I loaded everything on. So at this point with not much trouble I had an almost fully working setup. Even the webcam and sound was working.

    Video Fun

    The laptop is one of these strange modern ones that has two video cards.  The low-spec one is an Intel Sandybridge while the higher one is an AMD/ATI Radeon HD 6700M. The idea being you use the ATI when plugged in and the Intel on batteries.

    Xorg worked first go, but only on the Intel chip.  Trying to force X to use the ATI driver with an Xorg configuration resulted in the Xserver crashing.

    Next attempt was the fglrx packages within Debian. aticonfig said it didn’t know about the card. The reason for this is that the Debian packages use fglrx version11.4 and I needed for my card 11.5 So close but so far away.

    OK, so building the ATI software can’t be that hard can it?  A nice set of instructions on the Debian wiki and we had… errors. The first problem is that the packaging used is ancient. The second problem is much more serious; it won’t compile.

    I was stuck about here until I hopped onto the Debian irc channel and asked about it and someone found a very useful webpage about compiling the ATI drivers on Debian which importantly had information about the patches you need, especially the blk patch.

    Success! the program compiled and installed. A small xorg.conf specifying the fglrx driver and the BusID and we were up and running.  I now have accelerated graphics on the laptop.

    Ouch, my eyes!

    I do actually get outside (one of my hobbies is gardening and that is an outdoor activity) so I don’t need a monitor screen trying to give me a suntan.  The default screen brightness on the laptop could be described as “sub-nova”. I’m sure there is some alien astronomers several light-years away who will in many years time wonder what that flash from the unimpressive end of the milky-way was about. To protect your eyes and keep the neighborhood complaints down you can reduce the brightness to something sane. You do this in the power management settings which for me is System -> Preferences ->  Power Management. I can see again!

    Enhanced by Zemanta

  • JFFNMS 0.9.1 Released

    JFFNMS 0.9.1 has the database extracts and updates missing from 0.9.0  This is the most problematic part of the project release; ensuring that the database updates correctly.

    Version 0.9.1 is functionally the same as 0.9.0, it is just with less bugs!

     

  • What happens without software testing

     

    New TurboGears logo
    Image via Wikipedia

     

    Well JFFNMS 0.9.0 was a, well, its a good example of what can go wrong without adequate testing.  Having it written in PHP makes it difficult to test, because you can have entire globs of code that are completely wrong but are not activated (because the “if” statement proves false for example) and you won’t get an error.

    It also had some database migration problems.  This is the most difficult and annoying part of releasing versions of JFFNMS.  There are so many things to check, like:

    • What has changed between this version of the database?
    • Does the development database have all the new fields?
    • Will importing an old database and using the diffs give me the same result as importing the new database
    • Does the structure field match the object which matches the database table

    I’ve been looking at sqlalchemy which is part of turbogears.  It’s a pretty impressive setup and lets you get on with writing your application and not mucking around with the low-level stuff.  It’s a bit of a steep learning curve learning python AND sqlalchemy AND turbogears but I’ve got some rudimentary code running ok and its certainly worth it (python erroring on un-assigned varables but not forcing you to define them is a great compromise).  The best thing is that you can develop on a sqlite database but deploy using mysql or postgresql with a minor change.

    Python and turbogears both emphasise automatic testing. Ideally the testing should cover all the code you write.  The authors even suggest you write the test first then implement the feature.  After chasing down several bugs, some of which I introduced fixing other bugs, automatic testing would make my life a lot easier and perhaps I wouldn’t dread the release cycle so often.

     

    Related articles
    Enhanced by Zemanta

  • JFFNMS 0.9.0 Released

    After 3 release candidates JFFNMS is now at version 0.9.0. Both the web frontend and backend (engines) have had extensive re-work done to them to cleanup and tighten up the code. There should be a lot less warnings and errors from PHP when you set to a higher error level.

    • Fixed error in syslog consolidator which fails for postgresql
    • All webpage input passes through Sanitizer
    • register_globals no longer needs to be turned on, so turn it off!
    • rrdtool v1.0 support dropped
    • lots of cleanup
    • poller rewritten with parent/child code
    • autodiscovery the same as poller
    • Removed php rrd module support
    • Interface auto-discovery will check sysobjid before trying discovery
    • IPv6 reachability support
    • Separated interface selector code
    • SNMP interfaces can have High Speed and optinally no Cisco proxy ping
    Enhanced by Zemanta

  • JFFNMS – 3rd RC lucky?

    I’ve just uploaded to sourceforge the third and hopefully last RC for JFFNMS network management system version 0.9.0  The reason for the delay was easter as well as I wanted to test the engines for a long while to make sure I was not getting any orphan children or items. Previous versions had processes that never died or if they died the parent didn’t realise and didn’t handle the item, permanently making the item “out for work”.

    PHP5 has much better job and process handling and the new version takes advantage of this handling.  It’s run well on my on test setup for a week or two. You can find the RC code or just the older releases at https://sourceforge.net/projects/jffnms/files/

  • IPv6 and address translation

    We’re in that in-between place where there is plenty of IPv4 out there, but sometimes you either need or want to get to IPv6 addresses.  With the IPv4 address allocations out of ARIN now empty there is going to be an increased need of NAT either between the address families or within it.  This is a quick overview of the different sorts of systems that do this address translations.

    6tunnel

    The best way to describe this is port forwarding across the address families. You can have a IPv4 port on your local device that when you connect to it forwards the connection via IPv6 to a remote device, perhaps on a different port. You can also do it the other way, so connecting to a local IPv6 port will connect to a remote IPv4 port.  Of course “remote” could be another port on the same computer.

    Squid

    You could replace squid with any other sort of proxy server that can work as a reverse proxy and understands IPv6.  From the deployments I’ve heard of, this arrangement where the IPv4 only webservers are front-ended with a dual stack IPv4/IPv6 squid (or other) reverse proxy server or set of servers.  It is a real simple way of getting your webserver onto IPv6, if you cannot do it in the webserver itself.  The usual advantages and disadvantages of using a reverse proxy apply here.  The better long-term solution is to have the servers just run IPv6 natively, but this is a reasonable stop-gap.

    That’s a brief overview of the various packages out there that can help with address translation to either get your servers understanding IPv6 or a client to get to an IPv6 server.  Ideally these are only temporary measures but as some methods people use to write programs (the latest I’ve seen has weird hard-coded 4 byte offset tree-table with 256 long list inside – good luck fixing that!) they may be “temporary” for some time.

    Tayga

    This program implements NAT64 which is a 1:1 address translation.  It can work both ways so IPv6 clients can connect to IPv4 servers and vice-versa.  You would normally use this for your own hosts, rather than for connecting to the internet as you need to specify prefixes. It works like the old style static nat ranges in IPv4 where 1.1.1.10-100 would map to 2.2.2.10-100

    tnat64

    Now this package is a little back-to-front to the others in that it takes an existing IPv4 only application and makes it able to connect, at least at the socket level, to a IPv6 server.  It works by preloading a library which overloads the socket functions like socket() and connect() so it can attempt to find IPv6 servers.

    totd

    This is a small DNS proxy which can be used to translate IPv4 A records to IPv6 AAAA records.  You often use it with translation technologies like Tayga.

     

    totd

    This is a small DNS proxy which can be used to translate IPv4 A records to IPv6 AAAA records.  You often use it with translation technologies like Tayga.

    Related articles
    Enhanced by Zemanta

  • Flash sometimes doesn't like SSL

    I found a strange problem recently.  Using Gallery I could always upload new pictures but someone else using Vista could not upload any pictures.  It didn’t matter if she used IE or Firefox, every single upload made Gallery come up with error #2038 (IO). I could say that anyone that uses Vista deserves everything they get, but that would be a little churlish (if true).

    A few searches showed that this basically gallery saying “something bad happened with your upload”. No, really? You mean it’s not supposed to be a big red box and no uploaded file?  It’s not the world’s most useful error message.

    What got me onto the right path was a message on the gallery forums though it was the other way around. Gallery uses a flash uploader which and this is the same flash if you use Firefox or IE.  It seems it doesn’t like “strange” SSL certificates, such as chained certificates like that site uses (and my site too).  No doubt there is probably some work-around or setting to fix it, but being Vista its probably buried in 20 menu items under some obscure reference.

    Setting up the gallery under a non-ssl site fixed it. Perhaps more correctly it was a work-around.

    Enhanced by Zemanta