The other day I was browsing around on a professional networking site and came across a post in one of the forums from a guy asking for help on solving a monitoring problem. The problem was that when the servers got under a heavy enough load they would become unresponsive to his monitoring systems and the page would light up like a Christmas tree. He was looking for suggestions to relax the monitoring thresholds so the alerts wouldn't bug him.
/o\
The problem is not that the monitoring system is doing its job. The problem is that the servers are under such high enough load that they can't respond to simple "Are you still alive?" queries from a monitoring system. The correct solution is to either add hardware to the cluster and distribute the load or find some way to refactor the code so that it's more efficient.
More alarming was the fact that a bunch of people weighed in before I got there with various suggestions on how to increase timeouts, drop SNMP monitoring, etc. In a week, none of them said, "Hey, maybe the server being in distress is like, bad."
In the tech world, we fall prey to tunnel vision a lot. We become willing to push an incorrect solution to a problem so far that we will layer bad idea after bad idea on to a system, which in turn just keeps bringing in more and more points of failure. Pretty soon, you end up with a shaky, complex Akira style thing that is impossible for anyone else to understand or modify that does nothing but create unnecessary work for you.
The more I look around at my colleagues and talk with them about the battles they're fighting daily, the more I start to wonder if anyone else understands why the bearded Unixy elders held elegance in such high regard.
Monday, April 12, 2010
Wednesday, January 13, 2010
win32 c++
So I was presented with a situation where I needed a binary to simply run some predetermined commands on Windows. I said, "Why, this is a simple fork/exec under Linux. How hard could it be in Windows?"
Turns out, a lot fscking harder than it needs to be.
The primary problem stems from the way Windows "attempts" to support multibyte characters. Or rather, the wayS it supports multibyte characters... you have a choice of the ANSIish widechar that can do UTF-8, or the one that MS Visual Studio defaults to LPWSTR for UTF-16. The Windows libraries helpfully provide replacement functions for stuff like sprintf() and such, but the common and well known string functions are cut off from you. Worse, some wrappers to system calls like GetCurrentDirectory() will only return LPWSTR and only take LPWSTR args, and because these are differently sized types you can't just cast your out of the problem.
Oh, and stuff like malloc()/calloc() doesn't appear to work with the macros, either. Or at least I didn't have the time to unravel what the correct way to use them were in the time I had alotted.
In the end, I used something like:
wchar_t* thing[SIZE];
and just used a typecast whenever I needed to move into or out of the variable: (LPRWSTR) thing, like when using _stprintf() to build up these strings.
So, basically, I ended up being corralled into doing freshman level C just to get stuff working.
What I've taken away from this experience is that I absolutely, positively, 100% don not ever want to do any serious Win32 C/C++ development. Ever. EVER. And if I do have to go down that road, then I'm going to try and stick with ANSI... maybe even to the point of using cygwin exclusively to avoid getting sucked into this nightmare again.
Turns out, a lot fscking harder than it needs to be.
The primary problem stems from the way Windows "attempts" to support multibyte characters. Or rather, the wayS it supports multibyte characters... you have a choice of the ANSIish widechar that can do UTF-8, or the one that MS Visual Studio defaults to LPWSTR for UTF-16. The Windows libraries helpfully provide replacement functions for stuff like sprintf() and such, but the common and well known string functions are cut off from you. Worse, some wrappers to system calls like GetCurrentDirectory() will only return LPWSTR and only take LPWSTR args, and because these are differently sized types you can't just cast your out of the problem.
Oh, and stuff like malloc()/calloc() doesn't appear to work with the macros, either. Or at least I didn't have the time to unravel what the correct way to use them were in the time I had alotted.
In the end, I used something like:
wchar_t* thing[SIZE];
and just used a typecast whenever I needed to move into or out of the variable: (LPRWSTR) thing, like when using _stprintf() to build up these strings.
So, basically, I ended up being corralled into doing freshman level C just to get stuff working.
What I've taken away from this experience is that I absolutely, positively, 100% don not ever want to do any serious Win32 C/C++ development. Ever. EVER. And if I do have to go down that road, then I'm going to try and stick with ANSI... maybe even to the point of using cygwin exclusively to avoid getting sucked into this nightmare again.
kindle 2 weeks later
I still like what I said before about the Kindle, but the one thing it hasn't done is spurred my recreational reading. Yeah, I can rip through books quicker, but having trouble finding the time to actually commit to reading. Suspect this is work related, since I'm basically learning (or relearning) tech just about each day.
I tried out the magazine and newspaper subscriptions, but I found that even those weren't enough incentive to turn the sucker on each morning. I also found that periodicals who are publishing their content in Kindle format are playing this weak game of not including everything and instead reserving that for customers who have paid for subs to their website (or ordered the dead tree version). Same money, less content? No, but thanks anyway.
I'm also discovering that anything beyond new releases is going to be kind of hard to find kindlefied. An elusive book that has remained on my Stuff To Read list for way too long is When Harlie was One... but it's been out of print for nearly decades. The only way I'm likely to see it is if I can track it down at the library or maybe stumble across it in the Half Price Bookstore. Chances of getting via kindle (legally) are pretty low.
In other news, I've discovered that if you put the text to speech feature on it's fastest setting and use the female voice, it does a passable job of reading flowing prose to you at a rate that doesn't require too much attention. Was able to drive with it going on.
Bottom line: it's by no means perfect, but it's got enough bonuses to justify purchase. And those are mainly, 1) text to speech, 2) ability to convert your own material to kindle format, and 3) the fact that I won't have another 5 boxes of books to pack up.
I tried out the magazine and newspaper subscriptions, but I found that even those weren't enough incentive to turn the sucker on each morning. I also found that periodicals who are publishing their content in Kindle format are playing this weak game of not including everything and instead reserving that for customers who have paid for subs to their website (or ordered the dead tree version). Same money, less content? No, but thanks anyway.
I'm also discovering that anything beyond new releases is going to be kind of hard to find kindlefied. An elusive book that has remained on my Stuff To Read list for way too long is When Harlie was One... but it's been out of print for nearly decades. The only way I'm likely to see it is if I can track it down at the library or maybe stumble across it in the Half Price Bookstore. Chances of getting via kindle (legally) are pretty low.
In other news, I've discovered that if you put the text to speech feature on it's fastest setting and use the female voice, it does a passable job of reading flowing prose to you at a rate that doesn't require too much attention. Was able to drive with it going on.
Bottom line: it's by no means perfect, but it's got enough bonuses to justify purchase. And those are mainly, 1) text to speech, 2) ability to convert your own material to kindle format, and 3) the fact that I won't have another 5 boxes of books to pack up.
Tuesday, December 29, 2009
[books] ragamuffin
I'd heard about Tobias Buckell when he showed up on Dead Robots Society to do a podcast episode, and from the interview he sounded like exactly the kind of creative little monkey that makes stuff I like reading about. He'd mentioned a couple of his books, but the one that stuck out in my mind was "Ragamuffin".
As a result, that was the first I ended up snagging after I got my Kindle online.
It turns out that this is book two in a series that has 3 entries so far ("Crystal Rain" being the first, and "Sly Mongoose" being third) which means I've once again been introduced to a universe in the middle of the story. But the way the book is structured, you're eased into the history of the universe while being started off with some pretty brutal action in the story. I also really enjoyed the pace of the story and found it to be pretty well crafted overall.
I'd highly recommend this book, and have already snagged the other two in the series.
As a result, that was the first I ended up snagging after I got my Kindle online.
It turns out that this is book two in a series that has 3 entries so far ("Crystal Rain" being the first, and "Sly Mongoose" being third) which means I've once again been introduced to a universe in the middle of the story. But the way the book is structured, you're eased into the history of the universe while being started off with some pretty brutal action in the story. I also really enjoyed the pace of the story and found it to be pretty well crafted overall.
I'd highly recommend this book, and have already snagged the other two in the series.
now with 20% more kindle flavor
I scored a Kindle for Christmas, and have been busy catching up on a bunch of reading that I have neglected. Most of it is sci-fi stuff so far, but I'm also planning on ganking stuff from Project Gutenberg (as in "printing press", not "Police Academy") and some other interesting/educational texts.
The big hassle is that the Kindle's PDF reader is kind of limited. Specifically, it can't resize fonts at all, which makes reading some of the books I've got rather difficult. I've snagged Mobikit's Creator Publisher version and managed to convert both The US Army Ranger Handbook and all 24 of the NEETS modules into kindle format. Conversion seems to do great for text, but messes up formatting of stuff like tables and the placement of artwork.
Pretty slick stuff so far. Amazon did a good job of integrating their store with it and making it easy to acquire content.
The big hassle is that the Kindle's PDF reader is kind of limited. Specifically, it can't resize fonts at all, which makes reading some of the books I've got rather difficult. I've snagged Mobikit's Creator Publisher version and managed to convert both The US Army Ranger Handbook and all 24 of the NEETS modules into kindle format. Conversion seems to do great for text, but messes up formatting of stuff like tables and the placement of artwork.
Pretty slick stuff so far. Amazon did a good job of integrating their store with it and making it easy to acquire content.
Tuesday, December 15, 2009
ebox
Ok, so mentioned it briefly in another vanity tool that allows self-promotion 140 characters at a time, but I didn't get around to writing up anything on it here. Time to fix that.
At my orc place, we recently had a few folks move to Windows Vista (now 7) desktops at home. This was ok, until they attempted to connect to the VPN only to discover that it no longer worked. The problem is that Microsoft removed support for the MS-CHAPv1 protocol and only MS-CHAPv2 is available, but the VPN appliance we were using doesn't speak MS-CHAPv2. One option was to setup L2TP on the appliance, but that ended up being all kinds of fail due to some funky routing stuff that had to be preserved and was beyond my capabilities (the best I can do is smash the device with a hammer and swear a blood oath to track down all those responsible for producing it).
So. Fsck it all to hell. We started looking at Linux based solutions that could handle L2TP, and that's when I stumbled across OpenVPN (which supports Mac/Win/Lin). Easy enough to setup, but at the last minute got the added constraint that the solution should be generic enough that someone else could manage it (ie, the company could find a networking monkey to come in and make sense of it).
While getting ready to install it on an Ubuntu box, I noticed that they had "ebox-openvpn" and did some digging on that.
It turns out the eBox is one of those "appliance distros". It's based on ubuntu, and comes with a web config utility which seemed to satisfy the "normal people can use it" requirement. We ended up just downloading the distro that eBox offers up on their website and installing that without any major hassles.
eBox is actually a pretty cool little distro you should check out if you need a small intranet server for a hub office or a small, decentralized startup. It comes with not only a well laid out iptables system and OpenVPN, but also includes stuff like a mail virus/spam filter, file server, and ability to do the BC thing for active directory with samba/kerberos/ldap. Handiest of all, though, is that it's got a certificate management system that lets you setup your CA, and issue X.509 certs pretty easily.
There are some rough edges, though. Because it's an appliance distro, it's guilty of the same thing stuff like Plesk is guilty of. It considers its own internal database to be authoritative for all configurations on the system: if you hack a config file by hand, prepare for the changes to get blown away when the service or server is next restarted. That wouldn't be a problem, except for the fact that the web interface is by no means comprehensive when it comes to configuring the services. For example, we needed to add in some extra options to OpenVPN to tell it to force the client to set default route to the VPN, who the WINS and DNS servers, etc... fairly straightforward to do in the openvpn.conf, but there wasn't anyway to set those in eBox. Finally, my last gripe is that you have to "save" all changes before you can exit a config screen. This makes mass tweaking of the interface kind of tedious and slow going.
There was also a problem with the client openvpn package that ebox generated for us. The zip file had the right certificate, but the client didn't work. We ended up having to replace it with the .exe directly from openvpn's site.
As for solving the original problem, I'd suggest just nutting it up and learning how to deal with openvpn through it's config file. The config is actually pretty straightforward (and short). The main reason we decided to keep eBox installed and in place, however, was because of the certificate management feature and because we didn't want to spend anymore time re-installing a distro onto the server.
I'm not sure I'd recommend eBox for a large, more established network, but it definitely seems like the way to go if you need to get a small shop up and online in a couple of hours.
At my orc place, we recently had a few folks move to Windows Vista (now 7) desktops at home. This was ok, until they attempted to connect to the VPN only to discover that it no longer worked. The problem is that Microsoft removed support for the MS-CHAPv1 protocol and only MS-CHAPv2 is available, but the VPN appliance we were using doesn't speak MS-CHAPv2. One option was to setup L2TP on the appliance, but that ended up being all kinds of fail due to some funky routing stuff that had to be preserved and was beyond my capabilities (the best I can do is smash the device with a hammer and swear a blood oath to track down all those responsible for producing it).
So. Fsck it all to hell. We started looking at Linux based solutions that could handle L2TP, and that's when I stumbled across OpenVPN (which supports Mac/Win/Lin). Easy enough to setup, but at the last minute got the added constraint that the solution should be generic enough that someone else could manage it (ie, the company could find a networking monkey to come in and make sense of it).
While getting ready to install it on an Ubuntu box, I noticed that they had "ebox-openvpn" and did some digging on that.
It turns out the eBox is one of those "appliance distros". It's based on ubuntu, and comes with a web config utility which seemed to satisfy the "normal people can use it" requirement. We ended up just downloading the distro that eBox offers up on their website and installing that without any major hassles.
eBox is actually a pretty cool little distro you should check out if you need a small intranet server for a hub office or a small, decentralized startup. It comes with not only a well laid out iptables system and OpenVPN, but also includes stuff like a mail virus/spam filter, file server, and ability to do the BC thing for active directory with samba/kerberos/ldap. Handiest of all, though, is that it's got a certificate management system that lets you setup your CA, and issue X.509 certs pretty easily.
There are some rough edges, though. Because it's an appliance distro, it's guilty of the same thing stuff like Plesk is guilty of. It considers its own internal database to be authoritative for all configurations on the system: if you hack a config file by hand, prepare for the changes to get blown away when the service or server is next restarted. That wouldn't be a problem, except for the fact that the web interface is by no means comprehensive when it comes to configuring the services. For example, we needed to add in some extra options to OpenVPN to tell it to force the client to set default route to the VPN, who the WINS and DNS servers, etc... fairly straightforward to do in the openvpn.conf, but there wasn't anyway to set those in eBox. Finally, my last gripe is that you have to "save" all changes before you can exit a config screen. This makes mass tweaking of the interface kind of tedious and slow going.
There was also a problem with the client openvpn package that ebox generated for us. The zip file had the right certificate, but the client didn't work. We ended up having to replace it with the .exe directly from openvpn's site.
As for solving the original problem, I'd suggest just nutting it up and learning how to deal with openvpn through it's config file. The config is actually pretty straightforward (and short). The main reason we decided to keep eBox installed and in place, however, was because of the certificate management feature and because we didn't want to spend anymore time re-installing a distro onto the server.
I'm not sure I'd recommend eBox for a large, more established network, but it definitely seems like the way to go if you need to get a small shop up and online in a couple of hours.
Monday, November 16, 2009
moar laz0rs
So, got the table bits lashed together and working properly now. Ugly end product that looks like a 7th grader put it together:

And the end result is thus:

Unfortunately, one of the lasers is misaligned, so you still get bright light shining off your thumb when it's not touching the screen unless your hand is at a (completely unnatural) 90 degree angle to the table.
Support is the next thing to work on, then it's time to look at the projection stuff. Ouch.
What I ended up doing:
1) just using the clay to hold the laser and adjust the pitch and yaw, then striped some hot glue across the top to control roll.
2) stapled the wires up along the sides to the top of the frame
3) hacked up quickcam for the shots as noted in previous posts
It turns out that the grooves that I cut were pretty much unnecessary.. the light seems to shine fine if the laser is flush with the acrylic. We slapped wet modeling paper clay into each corner and used some guides to calibrate the beam, and pushed the laser down into the clay until it just about dug a channel out from the front of the laser to about half way back. Then we left the clay to dry for about 20 hours before gluing the lasers in place.
For calibration, we simply used 4 red Solo plastic cups to provide markers for the beam and adjusted the beams by using a webcam with the lights turned down low. We didn't use the modified logitech because I wanted to be able to see where the beam was in relation to the cup. Some stuff I learned during calibration:
1) if the beam appears to be V shaped, it means you've got one side angled up and the other side is a reflection from the table top.
2) the just above the white lip of the cups is about where you want the line to be, or right below the bottom crease on the cup
3) for the line splitters on the laser, you want to make sure that the rough side is facing towards the laser lens, and the smooth side is pointed out. :( Instead of a focused 89 degree arc, we got a 150 degree arc that dimmed the closer you got to the center
4) don't be afraid to drop the camera down under the table to make sure that you've got the laser shining at the right place... ie, test each laser from the under the table POV
And the end result is thus:
Unfortunately, one of the lasers is misaligned, so you still get bright light shining off your thumb when it's not touching the screen unless your hand is at a (completely unnatural) 90 degree angle to the table.
Support is the next thing to work on, then it's time to look at the projection stuff. Ouch.
What I ended up doing:
1) just using the clay to hold the laser and adjust the pitch and yaw, then striped some hot glue across the top to control roll.
2) stapled the wires up along the sides to the top of the frame
3) hacked up quickcam for the shots as noted in previous posts
It turns out that the grooves that I cut were pretty much unnecessary.. the light seems to shine fine if the laser is flush with the acrylic. We slapped wet modeling paper clay into each corner and used some guides to calibrate the beam, and pushed the laser down into the clay until it just about dug a channel out from the front of the laser to about half way back. Then we left the clay to dry for about 20 hours before gluing the lasers in place.
For calibration, we simply used 4 red Solo plastic cups to provide markers for the beam and adjusted the beams by using a webcam with the lights turned down low. We didn't use the modified logitech because I wanted to be able to see where the beam was in relation to the cup. Some stuff I learned during calibration:
1) if the beam appears to be V shaped, it means you've got one side angled up and the other side is a reflection from the table top.
2) the just above the white lip of the cups is about where you want the line to be, or right below the bottom crease on the cup
3) for the line splitters on the laser, you want to make sure that the rough side is facing towards the laser lens, and the smooth side is pointed out. :( Instead of a focused 89 degree arc, we got a 150 degree arc that dimmed the closer you got to the center
4) don't be afraid to drop the camera down under the table to make sure that you've got the laser shining at the right place... ie, test each laser from the under the table POV
Subscribe to:
Posts (Atom)