Thinking Considered Harmful

The Technical Musings of Aaron Meriwether

Recent Posts

Security | Meta

Goodbye Linode, Hello Hetzner - 25/Jan/2015

There’s an old saying in Tennessee — I know it’s in Texas, probably in Tennessee — that says, fool me once, shame on — shame on you. Fool me — you can’t get fooled again.
-_George W. Bush_

A long time ago I used to run Linux on a beige box at home as a webserver and router. Over the years, the box evolved, eventually being migrated to a VPS.

At first, I used Slicehost, and was very happy with them for a while, but eventually they were acquired by RackSpace and shut down.

At that point, I migrated to Linode, but they never quite felt as dedicated to doing things “right” as Slicehost had been. Over the next few years, they experienced several major security breaches, and still have not been able to explain to my satisfaction how these came about or what steps are being taken to avoid them in the future.

Additionally, the recent Snowden leaks call into question the integrity of all US-based providers with regard to engineered and incidental back doors for NSA access - if a back door is a known part of the system design, how secure can you really expect the system to be? Germany may not be as anti-NSA as Bolivia, but they are certainly not as buddy-buddy as the UK

So after the most recent Linode incident, I decided enough was......



Google Drive and Dropbox Sync - 19/Jan/2014

I also noticed that my harddisk … seems to be going, so keep your fingers crossed. I thought I’d better upload what I have now, rather than notice that I lost everything when I get back to work on Monday.. (Only wimps use tape backup: real men just upload their important stuff on ftp, and let the rest of the world mirror it ;)

Having lost many a hard drive myself, most recently just last month, I’ve spent a lot of time looking for a good, modern Internet-based backup solution. Years ago, Mozy began offering such a service, and I tried it out at the time, but for whatever reason, I didn’t keep using it. Other products have since appeared, such as Dropbox and Google Drive, and with the popularization of the concept of “The Cloud,” focus has shifted from a simple replacement for tape-backups, to a full document store. Features such as online document editing have also become common. Synchronization to multiple machines is also a becoming common use case.

Many of these products offer a free service tier that is sufficient for storing a reasonable amount of data - perhaps not a full disk image with applications, but at least my entire documents folder will fit. So the other day I decided to try out Dropbox and Google Drive. I signed up for a free account on each, and installed the free software on my......



Migrating To Jekyll - 15/Jan/2014

I’ve maintained this blog off and on for a few years now on a self-hosted WordPress instance, but that has felt a bit bloated and unwieldy to me. (Especially the awkward WYSIWYG post editor) so I’ve begin porting the blog over Jekyll, hosted on GitHub. It’s currently a work-in-progress, but I’ve made it far enough to push out an initial version while I work on migrating the remaining posts, porting the Disqus comments, tuning the CSS, and writing about the experience.

More to follow soon!!!



OSX as a PXE-Boot Server - 09/Nov/2012

I actually wrote this up a few months ago as a reply to a blog entry, detailing my own personal experience and variations on this process, but I’m reposting it here on my blog now for my own reference and for anyone else who is interested. This process can, of course, also be adapted to PXE-boot other things such as a CentOS Kickstart install.

Here are the steps I used to successfully PXE-boot OpenBSD from OSX. My MacBook Pro is connected to the Internet via the AirPort, and my soon-to-be OpenBSD box is connected to my Mac via the Ethernet port. As a slight added complication, my WLAN uses the 192.168.2.x subnet (which conflicts with the address range generally used by OSX’s Internet Sharing), so Internet Sharing needed to be adjusted to use a non-default address range.

So first, I fixed my Internet Sharing address conflict:

  • Disable Internet Sharing
  • Close any System Preferences windows
  • Edit /Library/Preferences/SystemConfiguration/ to add:

Then I configured and launched tftpd:

  • Edit /System/Library/LaunchDaemons/tftp.plist
  • Remove the Disabled key+value
  • Add -i to ProgramArguments
  • Invoke launchd:
    launchctl load -w /System/Library/LaunchDaemons/tftp.plist

I downloaded the appropriate OpenBSD pxeboot files to /private/tftpboot:

lwp-download lwp-download mv bsd.rd bsd

Edited the bootpd.plist file:

  • Launch Internet Sharing and make a copy of /etc/bootpd.conf.
  • Stop Internet Sharing and copy your bootpd.conf back to /etc
  • Edit more


pthread Names and IDs from Perl - 05/Sep/2012

Modern versions of Perl provide support for threads. On *nix systems, this is implemented via the system’s pthreads (AKA “POSIX threads”) support. This means that each thread looks to the operating system like its own lightweight process. POSIX threads on linux can run on separate cores, can have separate process ID and names, and can receive separate signals. Most of the existing Perl documentation isn’t very clear on how to manage these special attributes from Perl.

First of all, as per the perlvar manpage, it is possible to set the process name by modifying $0. In modern versions of Perl (>5.8), this affects two separate system properties: the process command-line, and the process name. The difference between these two properties is important to understand.

The process command-line is part of the original memory block the OS allocates when creating a process. This is usually used initially to pass the command-line options into the process, and is what populates @ARGV. After the process is started, it can overwrite this area with whatever it likes, and the result will be displayed by certain process-management tools such as ps and top. The size of this data is limited to whatever space the OS originally allocated for the *argv array. Because all of the threads in a single process share the same *argv area, there can be only one command-line across all threads of the process.

Additionally, Linux keeps some internal kernel metadata about each process,......



XHR, localStorage and Images - 09/Jul/2012

Traditionally, website resource management has been handled more or less opaquely by the browser. A webpage would declare a number of linked resources via src or href attributes on various HTML tags, and these would be fetched by the browser as it saw fit. Later, some control of resource caching was provided via server-side HTTP headers, but the browser remains the mastermind.

There are three main problems with this way of doing things. First, the caching of resources is not easily modified - once the browser has fetched a resource with a Cache-Control header defined, there is no telling if or when it will decide to refresh its copy short of the pre-defined TTL. Second, the order and progress of the resource fetching cannot be easily monitored or controlled. Finally, if you have used Cache-Control: no-cache, even though HTTP 304 responses may save you the bandwidth of re-downloading some large files, the browser still needs to fire off a separate request for every resource. Of course there are hackish work-arounds, such as appending a nonce to the query string to force a refresh, or attaching onLoad events to all img tags on the page, or using a spritesheets to bulk images together into a single HTTP request, but the usefulness of these is limited.

With recent technology developments, this no longer needs to be the case. For dynamic websites, the JavaScript XMLHttpRequest object provides the capability to fetch text-based resources from......



HEREDOC as a Normal File - 29/Jun/2012

The Bash HEREDOC feature is quite useful when you need to script the stdin input to a command, however not all commands can be coerced into reading their input from stdin. Some commands require that you supply filenames from which to read some of their input. In these cases, your Bash script could create a temporary file on-disk, write some content to it, execute the command, and then delete the temporary file afterword. This works of course, but wouldn’t it be nifty if there was a way to do this all at once via a HEREDOC? In fact there is, but it is not immediately obvious.

The standard HEREDOC syntax uses a double-left-angle-bracket to direct input to stdin. Like any other Bash redirection, this is just the standard form with the assumed target filehandle (0 aka stdin in this case). You can specify a different filehandle if you like. Filehandles 1 and 2 are stdout and stderr respectively and are thus not useful for input. But what about 3? Filehandle 3 doesn’t normally exist at process creation, but Bash can create it if you ask.

So we have something like this:

mycommand 3<<END my data here END

Well, that isn’t very useful in itself because the command probably isn’t aware that it should look at file descriptor 3 for input, but that’s where the next trick comes in: There is a directory on Linux under the /dev hierarchy which allows access......



Forcing HTTPS via mod_rewrite - 17/Feb/2012

A fairly typical question for Apache administration is how to force access to a certain set of resources to occur exclusively via HTTPS. This can be accomplished via browser-side JavaScript in a pinch, but the more canonical (and MitM-resistant) approach is to enforce this from the server side. Fortunately, this is fairly simple to accomplish using mod_rewrite rules in httpd.conf or an .htaccess file. For example, the below .htaccess file can be dropped into a directory, and assuming mod_ssl, mod_rewrite, and .htaccess files are all properly enabled, all requests for resources in this directory or its subdirectories will be forcibly redirected to the HTTPS version of the URL.

RewriteEngine On # redirect HTTP connections to HTTPS based on internal mod_ssl state RewriteCond %{HTTPS} !=on RewriteRule .? https://%{HTTP_HOST}%{REQUEST_URI} [R=301,L]

This doesn’t quite work in some more complex situations though. For example, in a configuration where the SSL layer is handled by a reverse-proxy/load-balancer like an F5 Big-IP, the built-in Apache HTTPS mechanism is useless because all connections arrive at the Apache box as HTTP. In this case, the HTTPS forcing could be accomplished with a rule on the proxy itself, but this can be complex to maintain since it places the configuration further from the data in question. Alternately, if the proxy has an option to enable the X-Forwarded-Proto header, you can still do the redirect at the Apache layer using......



XINS and NetBeans - 16/Feb/2012

There are many ways to build Web Services in Java. Probably the most common approach is to use JAX-WS annotations to build a SOAP web service within a J2EE container. Besides the reference implementation of JSR-224, the two web service frameworks from the Apache project, Axis2 and CXF, are the most well-known. When creating web services with any of these, WSDL documents are an important part of the process. They may either be generated from the API code after it is written (bottom-up), or they may be created first and used to generate a skeleton from which the API code development may begin (top-down). WSDL files are intended to be machine-readable, so although they provide an easy way to inform an application about available web service calls and parameters, they do a poor job of enlightening systems integrators as to the actual functionality behind the interface. In other words, a WSDL alone does not serve as adequate documentation. WSDL files are also tedious to construct by hand, so top-down approaches that start with a WSDL file are not always as nice as they sound. One other drawback to some WebService frameworks is that they focus only on the SOAP protocol, which works well for communication from purpose-built client applications, but is awkward from browser javascript (JSON or plain-old-XML are the usual preferred formats there).

So if......



Embedded Jetty Servlets and JSP - 03/Nov/2011

Recently I had been using Tomcat as a Java Servlet container for a project at work. This works well in the context of a tightly-integrated set of servlets and JSP pages like a typical website, but the project I was working on is intended to be a self-contained module which presents an HTTP API and should be easily deployable without worrying too much about shared settings and shared libraries on the target box. I also wanted the ability to profile, debug, and run jUnit tests on the component from within Eclipse, without requiring an additional, separate deployment to a Tomcat server (even if it is a local Tomcat server on my dev machine.

So I decided to switch to a design involving embedding a Jetty container into a plain Java app. This would give me the ability to have a clean deployable artifact with no external dependencies, and also to launch the servlet container and the HTTP tests against it from the same jUnit script. Sadly, the documentation for embedding Jetty is not the most comprehensive.

In order to keep things as tight and straightforward as possible, I opted to instantiate the individual parts of Jetty directly rather than just using the WebApp context as shown in the examples. (using the WebApp context would involve additional dependencies and overhead of dealing with WAR files and web.xml, etc.) Getting a basic servlet......



FreeBSD Framebuffer Graphics - 07/Jul/2011

I just pushed a new project up to GitHub. It’s the beginning of a console framebuffer graphics library for FreeBSD. While Linux has SVGALib, and BSD used to have VGL, there doesn’t seem to be anything current for this purpose, and the documentation for the kernel framebuffer ioctls is minimal (and even these seem mostly to just be thunks to the old interrupt 10h functions). I need this for a certain BSD-based side project, and since no one else seems to have it working at the moment I had to do it myself.



JavaScript Function Chaining - 26/May/2011

From time to time when working with JavaScript, I come across situations where I want to extend some existing function. Sometimes this function is defined in a file which I cannot or do not wish to change - it is often a bad idea to fork a local modified copy of a library which is maintained by someone else upstream who might release a new version. You would then be forced you to manually port your hack into their new version of the source code (though a good source control mechanism can be very helpful there). In these cases, function chaining is an option.

Though it may not appear so initially, JavaScript actually excels at this sort of thing - and that is good since JavaScript is often employed to patch incremental functionality on top of existing code. Lets disregard for the moment the advent of newer, more advanced DOM mechanisms like addEventListener and hark back to ye olde days of window.onload. Something like this perhaps:

/* * prettytitle.js - Makes the titlebar into an image. */ // wait until the document has finished loading. window.onload = function() { // get the titlebar element and replace its content with an image tag. document.getElementById('titlebar').innerHTML = '<img src="titlebar.png" />'; };

Then say this website needs to add another, separate progressive-enhancement script. Or two. Or......



Laziness - 23/May/2011

One of my friends has a habit of posting obfuscated strings as Facebook comments when he gets bored. Being lazy, I don’t usually feel like spending my time doing things like converting binary to decimal or hex



Discovering Node.js - 23/May/2011

Sometime last year I came across Node.js, the server-side port of Google’s V8 ECMAScript engine. At first I was interested in it mostly as a novelty to allow things like form validation library portability from client to server side. After trying it out and reading through the documentation in more detail and watching Ryan’s presentation, I became a lot more excited about it.

It turns out that Ryan Dahl has taken the task of developing a server-side JavaScript environment, which would seem at first blush to be something unholy, bulky and perhaps even passe, and reframed it in a way that really showcases JavaScript’s strengths and builds on the single-threaded event model pioneered by the DOM and familiar to browser-savvy developers the web over. Some might find the lack of threads a complete put-off, but having worked with Boa in the past, I was immediately attracted to this idea.

Since Node.js is essentially just a set of bindings for V8 to allow it to interact with a server type environment instead of the more familiar DOM, Ryan had complete freedom to implement all of the core I/O API calls in a way that relies exclusively on callbacks to avoid blocking. It is this clever trick that allows Node.js to perform extremely well in unexpected roles such as webserver, while at the same time avoiding the complex select event loop logistics usually associated with single-threaded daemons. This......