Dec 27 2014
3 Comments
RaspberryPi

Fix USB Serial Console on Raspberry Pi for Yosemite

My favorite way to talk with Raspberry Pi devices is using the PL2303 based USB Serial cable from Adafruit. The convenience of not needing a monitor or keyboard, as well as not needing a network connection for SSH, is significant as I often want to work with my devices in varied environments, and have several devices deployed “in the wild” where other options would be very inconvenient.

The Problem

Unfortunately this functionality recently stopped working for me. A number of variables had changed; I had upgraded my laptop to OS X 10.10 (Yosemite), I was using a new USB Serial cable I hadn’t used previously, and I was using a Model B+ Raspberry Pi which I hadn’t worked with before. As anyone who’s done much debugging will be all top familiar with, having this many variables change creates a nightmare for debugging, particularly as I had no available hardware to put any of the variables back to a known-good state.

After a lot of digital multi-metering and loopback testing I was convinced the Raspberry Pi was functioning correctly, so I was able to turn my focus to the USB serial adapter itself. I had used these cables successfully in the past, so I had faith in the concept, but for some reason when I attempted to use one using screen (screen /dev/cu.PL2303-00001014 115200) with my Raspberry Pi I merely received a blank screen with a blinking cursor.

When I installed the drivers for the PL2303, I had followed a link from this page in Adafruit’s article on using the cable to a package installer. Returning a saw a large amount of discussion around issues with using the drivers on Yosemite, with a number of suggestions for fixing the issue. I tried all of the suggested fixes, removing and reinstalling the driver using the supplied package several times, without positive results. At some point during the process I discovered that as long as the cable was connected during boot it would function properly, but as soon as it was removed and replugged into a USB port while the computer was running it would give me the black screen, and the following errors would show in my dmesg log:

➜  ~  cat dmesg | grep PL2303
nl_bjaelectronics_driver_PL2303(0xffffff801da83200)::dequeueDataGated - INTERRUPTED
nl_bjaelectronics_driver_PL2303(0xffffff801d55d800)::allocateResources failed - no fpInterface.
nl_bjaelectronics_driver_PL2303(0xffffff801d55d800)::start Allocate resources failed
nl_bjaelectronics_driver_PL2303(0xffffff801d55d800)::CheckSerialState - StartSerial failed
nl_bjaelectronics_driver_PL2303(0xffffff801d55d800)::enqueueDataGated fTerminate set
nl_bjaelectronics_driver_PL2303(0xffffff801d55d800)::enqueueDataGated fTerminate set
nl_bjaelectronics_driver_PL2303(0xffffff801d55d800)::enqueueDataGated fTerminate set
nl_bjaelectronics_driver_PL2303(0xffffff801d55d800)::enqueueDataGated fTerminate set
nl_bjaelectronics_driver_PL2303(0xffffff801d55d800)::enqueueDataGated fTerminate set
nl_bjaelectronics_driver_PL2303(0xffffff801d55d800)::enqueueDataGated fTerminate set

The Solution

Finally I decided to try giving up on the package installer and going to the manufacturers driver page here. Fortunately the manufacturer now actually supplies it’s own package installer. I removed the existing install of the driver using these steps:

  1. Disconnect the USB cable from the computer
  2. Run these commands: 
    sudo kextunload /System/Library/Extensions/osx-pl2303.kext
sudo rm -r /System/Library/Extensions/osx-pl2303.kext
  3. Remove any instances of “USB Serial Controller …” from Network Preferences.

I then used the installer from the above link to prolific’s website, and then rebooted.

The device identifier changed from it’s previous value of /dev/cu.PL2303-00001014 to a more intuitive /dev/cu.usbserial, but more importantly I was able to consistently connect to my Raspberry Pi, even without the cable connected during my laptops boot:

➜  ~  screen /dev/cu.usbserial 115200


Raspbian GNU/Linux 7 raspberrypi ttyAMA0

raspberrypi login:
Tagged , , , , , , , , ,
Nov 22 2014
Leave a comment
Web Development

Develop Against External Webhooks Locally Using Ngrok

I wanted to do some work developing some functionality against Github using Hubot, and I hit the classic challenge of attempting to develop against a webhook-centric interface: “How am I going to simulate the webhooks?”.

Fortunately I did some basic searching and stumbled across a project called Ngrok. Ngrok is a pay-what-you-want tool that allows you to expose a local server externally without all the headaches that would otherwise be necessary at the network infrastructure level.

Below follow step-by-step instructions on how to get up and going with Ngrok. Assuming your an on an OS X development machine. If you’re using another operating system most of the steps will be close but details will most likely vary.

Step 1. Sign Up With Ngrok (optional)

Incredibly it’s not actually necessary to sign-up for an Ngrok account to use their tool. If you don’t wish to signup you can simply proceed to Step 2., however, there are some nice features that are available if you do signup, such as custom subdomains and password protected tunnels. These instructions will assume you have signed up for an account, so if you skip this step your mileage may vary.

Simply proceed to https://ngrok.com/user/signup and enter your details, after signup you will be prompted to pay (only if you wish!) and then should eventually end up on a dashboard that looks something like this:

Step 2. Download Ngrok

Click on the downloading ngrok link on the dashboard (or here):

Download Ngrok

Choose the appropriate version of ngrok for your operating system:
Ngrok Download OS X

After downloading I chose to move ngrok to my Applications folder. This is not necessary, but if you place it in a different location adjust the rest of the instructions accordingly:
Move Ngrok to Applications

Step 3. Starting Ngrok

WARNING: This will expose your local port 80 to the internet. Exercise with Caution!

Simply run the command that is shown on your dashboard in a terminal to start Ngrok. It will look something like this:

ruby-2.1.2 ➜  ~  /Applications/ngrok -authtoken  80

The authtoken is only necessary the first time you start Ngrok as it remembers it in a ~/.ngrok file for you.

After starting ngrok, you should see something like this in your terminal:
Ngrok Terminal

Step 3. Testing Ngrok

Ngrok creates a web interface that’s available on port 4040, and can be used to inspect packets. You can checkout it out by going to http://localhost:4040 in your browser. You should see something like this:
Ngrok Web Dashboard

Ngrok has created both an http and https endpoint on the external web for your local Port 80. You can check out what happens by clicking on either link. For me, I currently have the default Apache “It works!” page running on my development machines port 80, so I see this:

56b49aac_ngrok_com

So at this point I now have a tunnel open that’s redirecting an external url to a local server. To start developing against those webhooks you simply need to give them that Ngrok URL as your webhook or call back endpoint, and start building!

But wait, there’s more!

Step 4. Checkout some other cool features!

Ngroks web interface will keep a history of all the requests it proxies, including request times, request/response header, body.

ngrok

It even has a “Replay” button, so you can retransmit any requests to your server. This means you can re-test any callbacks you want easily without having to drive the 3rd party system you are building against. This is a amazing feature, which will make development significantly faster by saving you the hassle of clicking-driving external products:

Resubmit Button

Conclusion

This is one of the coolest development tools I’ve seen this year, and on a pay-what-you-want model its an amazing value. If you try this out and get value from it, I highly recommend contributing back to the project. They accept paypal, so it’s easy to throw a few dollars their way. I personally think it’s well worth the recommended $25 yearly contribution.

Tagged , , , , , , , , , , ,
Nov 17 2014
Leave a comment
Vim

More permanent trailing whitespace highlight in vim

I’d had an issue with my Vim config where when I started a new instance of Vim my trailing whitespace highlighting would work, but then after a period of time it would stop working. This had bothered me for a while, but I finally decided to just dedicate the time and figure out what was happening.

When I looked at how it was implemented in my .vimrc file, I found a solution that is commonly suggested, such as in this stackoverflow post, or the first solution at this wikia post. Basically it looked like this:

" Highlight trailing whitespace
highlight ExtraWhitespace ctermbg=darkgreen guibg=darkgreen
match ExtraWhitespace /\s\+$/

The first `Highlight` command sets up the colors that I want to use to highlight the whitespace, and the second `match` command setups up a regex to identify the whitespace, and binds it to the `ExtraWhitespace` highlight group.

There’s a problem with using the `match` command like this though. The `match` command is generally meant to be used for more temporary purposes (for example if you only temporarily wanted to highlight your trailing whitespace). It is only ever possible to track 3 matches at a time (using `match`, `2match`, and `3match` respectively), and it would be common for other parts of your vim configuration, or 3rd party plugins, to overwrite this configuration for their own purposes.

There is a better way! By using syntax highlighting along with an autocmd, we can perform a similar match, but in a much more persistent manner. This is what I modified my .vimrc to be now:

" Highlight trailing whitespace
highlight ExtraWhitespace ctermbg=darkgreen guibg=darkgreen
autocmd Syntax * syn match ExtraWhitespace /\s\+$/ containedin=ALL
Tagged , , , , , , , ,
Oct 18 2014
Leave a comment
Ruby

Rails Composer – Easily setup a new rails project

I can’t even count the number of rails projects I’ve created over the last five years. For a while I’ve been using the site RailsWizard to take away some of the drudgery of setting up the typical gems I like to work with. Recently someone introduced me to an alternative called Rails Composer.

If you end up on their website it can be difficult to figure out that the functionality they offer is actually free. You can find it, however, if you go to the projects repo page on GitHub. The money line is this:

$ rails new myapp -m https://raw.github.com/RailsApps/rails-composer/master/composer.rb

I suggest you give it a shot. So far it looks really promising to me.

Tagged , , , ,
Sep 06 2014
Leave a comment
Uncategorized

Life is Pursuit

Karl Marx said ‘Life is struggle’. I believe that this misconstrues the point. Life is pursuit, and it is in pursuit that we find struggle. You cannot have pursuit without struggle, but to focus on the struggle is masochistic.

Jul 09 2014
Leave a comment
RaspberryPi

Serial Port Sees Kernel Sequence But No Login Prompt In Raspbian

I just finished fighting a battle debugging the serial console on the raspberry pi. I was able to see the kernel boot sequence, so I was confident there were no pin-out issues, but following the kernel sequence I wasn’t being presented with a login prompt:

TLDR;

If you have this problem, first check that the following line is present and uncommented in your /etc/inittab file:

#Spawn a getty on Raspberry Pi serial line
T0:23:respawn:/sbin/getty -L ttyAMA0 115200 vt100

If you’ve installed upstart on your Raspberry Pi, you must add the following file to /etc/init

/etc/init/ttyAMA0.conf

# ttyAMA0 - getty
#
# This service maintains a getty on ttyAMA0 from the point the system is
# started until it is shut down again.

start on runlevel [23] and not-container

stop on runlevel [!23]

respawn
exec /sbin/getty -L ttyAMA0 115200 vt100

THE DETAILS

I have a project that’s going to require me to configure the wifi on a Raspberry Pi in place. This made me nervous, as the only interface I would have available into the device was through WiFi, so a mistake in configuration would effectively brick the device.

Looking for a safer solution I stumbled upon the Adafruit USB to TTL cable. This cable allows you to access the serial terminal available through the pins on the Raspberry Pi header directly from the USB port of a computer without the need of other serial to usb converters. This is due to the embedded PL2303 chip in the cable that performs the conversion.

After following the instructions in this guide on Adafruit’s site, I was able to see the linux kernel boot sequence in a terminal on my mac:

...
[    9.614251] usbcore: registered new interface driver snd-usb-audio
[    9.716471] ieee80211 phy0: rt2x00_set_rt: Info - RT chipset 5390, rev 0502 detected
[   10.572707] FAT-fs (mmcblk0p5): Volume was not properly unmounted. Some data may be corrupt. Please run fsck.
[   11.397452] ieee80211 phy0: rt2x00_set_rf: Info - RF chipset 5370 detected
[   11.567387] smsc95xx 1-1.1:1.0 eth0: hardware isn't capable of remote wakeup
[   11.903032] usbcore: registered new interface driver rt2800usb
[   12.114566] init: failsafe main process (292) killed by TERM signal
[   13.929262] ieee80211 phy0: rt2x00lib_request_firmware: Info - Loading firmware file 'rt2870.bin'
[   13.986951] ieee80211 phy0: rt2x00lib_request_firmware: Info - Firmware detected - version: 0.29
[   14.741621] init: udev-fallback-graphics main process (414) terminated with status 1

Unfortunately, after this I was supposed to see a login prompt, and no amount of waiting was seeing one appear. I logged into the Raspberry Pi, and did a quick ‘ps aux | grep getty’ to look for the getty process that is supposed to create the console on the serial port. The default getty processes were present, but I should have seen a process using the device ttyAMA0, and it wasn’t present:

pi@raspberrypi ~ $ ps aux | grep getty
root       664  0.0  0.1   3744   804 tty4     Ss+  02:03   0:00 /sbin/getty -8 38400 tty4
root       666  0.0  0.1   3744   804 tty5     Ss+  02:03   0:00 /sbin/getty -8 38400 tty5
root       671  0.0  0.1   3744   804 tty2     Ss+  02:03   0:00 /sbin/getty -8 38400 tty2
root       672  0.0  0.1   3744   804 tty3     Ss+  02:03   0:00 /sbin/getty -8 38400 tty3
root       674  0.0  0.1   3744   804 tty6     Ss+  02:03   0:00 /sbin/getty -8 38400 tty6
root       980  0.0  0.1   3744   804 tty1     Ss+  02:03   0:00 /sbin/getty -8 38400 tty1
pi        1912  0.0  0.1   3520   796 pts/0    S+   02:21   0:00 grep --color=auto getty

I started doing some digging, and all the relevant threads I could find kept on telling you to ensure that the relevant line used to launch getty on the serial port was uncommented like this:

/etc/inittab

# /sbin/getty invocations for the runlevels.
...

#Spawn a getty on Raspberry Pi serial line
T0:23:respawn:/sbin/getty -L ttyAMA0 115200 vt100

No matter how many times I checked, that last line was uncommented in my file, but my login prompt continued to fail to appear.

After lots of beating my head against the wall, I finally realized that earlier in setting up the environment I had installed Upstart on my installation of Raspbian. Upstart replaces the inittab script, and instead uses .conf files located at /etc/init. Looking at the .conf files, there was no file to start the getty process I needed for the serial port.

I added the following to /etc/init/ttyAMA0.conf:

# ttyAMA0 - getty
#
# This service maintains a getty on ttyAMA0 from the point the system is
# started until it is shut down again.

start on runlevel [23] and not-container

stop on runlevel [!23]

respawn
exec /sbin/getty -L ttyAMA0 115200 vt100

With this new .conf file in place I rebooted my pi, and a login prompt like the following:

...
[   13.771014] ieee80211 phy0: rt2x00lib_request_firmware: Info - Loading firmware file 'rt2870.bin'
[   13.844754] ieee80211 phy0: rt2x00lib_request_firmware: Info - Firmware detected - version: 0.29
[   14.282944] init: udev-fallback-graphics main process (405) terminated with status 1
��
Debian GNU/Linux 7 raspberrypi ttyAMA0

raspberrypi login:

Finally my serial login was working!

Tagged , , , , , , , , ,
May 25 2014
Leave a comment
OS X

Selecting Airplay Speakers in OS X Has No Effect

It appears to have started occurring after I upgraded to Mavericks (OS X 10.9), though I can’t confirm this for sure. When I select an airplay output device from my Macbook Air nothing happens.

TLDR;

If you have this problem restart the music service for your mac with the following command:

sudo kill `ps -ax | grep 'coreaudiod' | grep 'sbin' |awk '{print $1}'`

THE DETAILS

If I hold the option key and click on the speaker icon in the menu bar, I see the Airplay devices available on my network and am able to select them:

However, after selecting a device the audio will continue to play from my Macbook Air, and if I re-open the menu under ‘Output Device:’ my ‘Internal Speakers’ are still selected.

Looking for a solution to this problem I came across this forum thread which suggested the following command, which I break down below:

##### Full Command #####
sudo kill `ps -ax | grep 'coreaudiod' | grep 'sbin' |awk '{print $1}'`

##### Breaking down the command #####
# Lists the running processes on your computer
ps -ax                                   
# Finds the line for the core audio process in the process list
ps -ax | grep 'coreaudiod'       
# Avoids the problem where the grep process also shows up in the ps results        
ps -ax | grep 'coreaudiod' | grep 'sbin' 
# Pulls out the Process ID for the core audio process
ps -ax | grep 'coreaudiod' | grep 'sbin' |awk '{print $1}' 
# Uses the Process ID of the core audio process to kill it (it will auto-restart)
sudo kill `ps -ax | grep 'coreaudiod' | grep 'sbin' |awk '{print $1}'`

This command has been a life saver for me, though I’m still not sure why Apple’s core audio service would be so brittle to get into the bad state necessitating this command so frequently.

Tagged , , , , ,
May 25 2014
Leave a comment
Ruby

Create RVM gemset along with .ruby-version and .ruby-gemset

Usually when I start a new project I want to create a new gemset along with the .ruby-version and .ruby-gemset files in the project directory. Inevitably I’ve since forgotten the syntax and spend 5 or 10 minutes trying to track it down. To save me that time I’m going to record the command here so I know where to find it next time I need it:

➜ rvm --ruby-version use [ruby version]@[gemset name] --create
Tagged , , , ,
Mar 31 2014
Leave a comment
PHP

Strtotime, PHP, and the End of the Month

Some of our PHP tests at Mover started failing today. In particular they were tests around our account system looking at the start and expiration dates for our plans:

1) Authenticator::->subscriptionStartDate::should be close to now
expected 1396389125 to be within 1396302607..1396302727

Some quick calculation shows that the date returned in the tests was 86518 seconds outside of the expected window, coincidentally close to the number of seconds in a day (86400).

It turns out this is a fairly widely discussed complication of the strtotime function in PHP, which we use to calculate the expiration and renewal periods for our plans. Our code to calculate the expiration date for a plan looks something like this:



Then later when we want to generate the subscription start date we subtract 1 month from the expiration date (why store redundant information in your database?)


$start_time =  strtotime(date(\DateTime::W3C, $expiration) . " - 1 month");



This, however, can yield some unexpected results:


$now
2014-03-31T21:58:43+00:00

($now + 1 month)
2014-05-01T21:58:43+00:00

($now + 1 month) - 1 month
2014-04-01T21:58:43+00:00



This is because of how PHP's strtotime() function handles adding one month to a date for which the date does not exist in the next month.  For example, if someone subscribed on January 31st, strtotime() will tell me that 1 month after January 31st is March 3rd.  Personally this logic surprised me (and apparently many others).  Fortunately we are already in the process of porting our account system to Recurly, which handles this problem in a very intuitive way.  From Recurly's FAQ:



Customers are always charged on the closest corresponding date of the following month. For example, a customer who would normally be billed on the 31st of a month will either be billed on the 30th of a 30 day month, or on the 28th of February.
Tagged , , , , , ,
Mar 20 2014
Leave a comment
Wordpress

WordPress PHP Files Downloading Instead of Processed by Server

I was helping a friend who was migrating a WordPress site from one host to another. After moving the files over to the new hosting provider instead of the files being server as PHP files by the web server a download prompt would pop-up to download a file. It turns out this was caused by the .htaccess file from the initial host (that was transferred along with the other files) not being properly configured for the new host. Here’s the initial .htaccess file:

# Use PHP5.4 as default
AddHandler application/x-httpd-php54 .php

# BEGIN WordPress

RewriteEngine On
RewriteBase /
RewriteRule ^index\.php$ - [L]
RewriteCond %{REQUEST_FILENAME} !-f
RewriteCond %{REQUEST_FILENAME} !-d
RewriteRule . /index.php [L]


# END WordPress

Note the AddHandler directive at the top of the file. My attention was drawn to it as the rest of the .htaccess file are the basic directives required by WordPress. This was likely added by the initial host to default .php files to be handled by PHP 5.4. The new host didn’t have support for PHP 5.4, which caused .php files to be mishandled. The issue was resolved by commenting out the line:

# Use PHP5.4 as default
# AddHandler application/x-httpd-php54 .php

# BEGIN WordPress

RewriteEngine On
RewriteBase /
RewriteRule ^index\.php$ - [L]
RewriteCond %{REQUEST_FILENAME} !-f
RewriteCond %{REQUEST_FILENAME} !-d
RewriteRule . /index.php [L]


# END WordPress

With this change the site began to function properly.

Tagged , , , , , , ,
← Older posts