Article RSS Feed

Spy on external HTTP API requests in Rails

Mon, 16 Nov 2009 04:18:00 GMT

Ever had to use a third party API through a gem or library in your Rails app and wondered where the requests are going to?

Through the magic of Net HTTP Spy (a little gem I wrote) you can spy on the majority of HTTP traffic to external API’s taking place in your Rails application.

Installation

sudo gem install martinbtt-net-http-spy

Real World Example using Google Data API’s

You can now get access to pretty much any Google application/content through their comprehensive set of API’s - very handy indeed.

In this example we are going to use the DocList manager sample application as seen on the GData on Rails tutorial pages.

First grab the source:

svn co http://gdata-samples.googlecode.com/svn/trunk/doclist/DocListManager

Then create a new file initializers/api_logging.rb and add the following:

if ENV.has_key?('API_LOGGING')
  gem 'martinbtt-net-http-spy'
  require 'net-http-spy'
  Net::HTTP.http_logger = Rails.logger
  Net::HTTP.http_logger_options = {:body => true} if ENV['API_LOGGING_FULL']
end

Sample Usage and Output:

API_LOGGING=true ./script/server
Processing DoclistController#documents (for 127.0.0.1 at 2009-11-14 20:24:47) [POST]
  Parameters: {"authenticity_token"=>"76e735433a9459f28a2dfb23090dec203b73e65c"}
CONNECT: ["docs.google.com", 80]
GET /feeds/default/private/full/-/document/mine
PARAMS {} 
BODY: Net::HTTPOK
Rendered doclist/documentslist (7.4ms)
Completed in 1672ms (View: 11 | 200 OK [http://localhost/]

Processing DoclistController#show (for 127.0.0.1 at 2009-11-14 20:24:49) [POST]
  Parameters: {"url"=>"http://docs.google.com/feeds/default/private/full/document:0BUjxlHAOrFICZGh1aGdrZ18xM2ZudGt3M2Zz", "authenticity_token"=>"76e735434a9459f08a2dfb23090dec203b73e65c"}
CONNECT: ["docs.google.com", 80]
GET /feeds/default/private/expandAcl/document:0BUjxlHAOrFICZGh1aGdrZ18xM2ZudGt3M2Zz
PARAMS {} 
BODY: Net::HTTPOK
CONNECT: ["docs.google.com", 80]
GET /feeds/download/documents/Export?docID=0BUjxlHAOrFICZGh1aGdrZ18xM2ZudGt3M2Zz&exportFormat=png
PARAMS {} 
BODY: Net::HTTPOK
Rendered doclist/_show (7.7ms)
Completed in 2515ms (View: 15 | 200 OK [http://localhost/doclist/show]

Note 1: By setting API_LOGGING=full the BODY: portion of the logging output will be replaced with the actual body response from the server. In this case it is the XML used to display the documents.

Note 2: Not all API libs are built the same and many interact with Net::HTTP in ways not easy to capture. Also be warned.. this modifies the Net::HTTP class directly so be careful if you intend using this in production.

Summary

Net HTTP spy provides a great way to peek at what is going on under the hood of your application and find out which http requests are being made to third parties. I’ve used this technique successfully with my team at work to track down API issues and I’ve also had numerous reports of other developers making use of the gem for the same reasons.

I hope you find this post helpful and if you have any further questions or feedback feel free to ask in the comments section.

Quick Reference

HTTP Spy on GitHub: http://github.com/martinbtt/net-http-spy
GData on Rails tutorials: http://code.google.com/apis/gdata/articles/gdata_on_rails.html
Google API Directory: http://code.google.com/apis/gdata/docs/directory.html

Edge Monkeybars

Thu, 04 Dec 2008 14:27:00 GMT

Monkeybars lets you create elegant looking cross platform desktop apps with JRuby.

You’ll want to play with the latest and greatest versions of Monkeybars without conflicting with your current gem install. Here is a little guide on how to do just that.

Download

Fire up a terminal and enter the following:

mkdir -p ~/code/gems
git clone git://gitorious.org/monkeybars/mainline.git monkeybars-mainline
cd monkeybars-mainline
rake jar
mate ~/.bash_profile

This might take a while so grab a cup of coffee while you wait. At last count the Monkeybars repo was ~90MB.

Edit your bash profile

Once your bash profile pops up in Textmate add the following to the bottom of it:

function monkeybars-edge() { 
 ruby ~/code/gems/monkeybars-mainline/bin/monkeybars $1 $2
}

Great, almost there. You’ll just need to reload you bash profile, so either start a new terminal or run the following

. ~/.bash_profile

Note the leading dot.

Result?

Lets see if that worked:

martin$ monkeybars-edge myapp
Creating directory myapp
Copying monkeybars project structure

Excellent.

Anything else?

The version of Monkeybars added to the classpath in src/manifest.rb is likely to be an older one. If you peek in the lib/java directory you’ll see which version you have. Update the manifest line accordingly. In the example below it was 0.6.4 and I updated to 0.6.5.

require 'resolver'

case Monkeybars::Resolver.run_location
when Monkeybars::Resolver::IN_FILE_SYSTEM
  add_to_classpath '../lib/java/monkeybars-0.6.5.jar'
end

RailsConf Europe presentation slides now published

Mon, 08 Sep 2008 15:45:00 GMT

The presentation files from the talk I gave at RailsConf Europe are now available for download.

Direct link: Stories on a Cloud - Distributed Browser Testing with Selenium Presentation (PDF format).

I really enjoyed giving the talk and their seemed to be enough interest for follow up material. All good! I’ll be publishing supporting links and research details very soon, so watch this space.

Speaking at RailsConf Europe

Tue, 02 Sep 2008 08:52:00 GMT

A quick shout about our talk tomorrow at RailsConf Europe:

Drive your browser (or somebody else’s!) through an RSpec story. In this talk I’ll be covering Rspec stories, Selenium, message queues, packaging your apps for cross platform distribution, and more!

Read more info over at the O’Reilly Conference page

Is this thing on? Converted to Radiant from Mephisto

Tue, 29 Jul 2008 15:16:00 GMT

More news to follow shortly..

sneak peak: http://github.com/martinbtt/radiant-import-mephisto

New RSS feed URL

Thu, 03 Jan 2008 21:00:00 GMT

Just a quick note to say the RSS feed for this blog has changed. Please update your feed reader to use:

http://feeds.feedburner.com/BeyondTheType

New Year, New start. Hello CitySafe

Wed, 02 Jan 2008 21:05:00 GMT

As of January this year I’m pleased to announce I will be working with the team at CitySafe.

CitySafe build tools to help emergency responders collaborate and communicate. These include mission critical apps in use daily by the Police, Government, and high profile financial and retail institutions. It’s predominately Ruby (on Rails) based so I’m looking forward to getting stuck in.

For those of you wondering about Working With Rails…. This remains under ownership of DSC and I have passed all the day to day running over to colleagues there. Prior to my departure I ensured that the Hackfests would continue up until March. Much thanks goes to Josette @ O’Reilly for the RailsConf ticket prizes.

All in all WWR has been an amazing project. I’m so pleased to have created it and see it grow from strength to strength benefiting the community so greatly. Many thanks to everyone who uses the site and contributed in some way over the past year. It has been fantastic to have your input and feedback.

I’ve heard many success stories from developers starting user groups as a result of finding each other on WWR, to getting a job through recommendations, collaborating on projects or helping to get Rails adopted in their workplace. Not forgetting to mention the numerous prizes given away through the regular Hackfests.

I hope it has benefited you in some way and continues to do so. Thanks everyone!

Stay tuned…

Continue to keep up with my news via this blog (rss) and via Twitter

VirtueDesktops alternative for OS X 10.5 (Leopard)

Wed, 21 Nov 2007 10:51:00 GMT

Just upgraded to Leopard and sad at the loss of VirtueDestkops? Spaces not doing it for you? Read on…

Background

When Leopard was announced Spaces was touted as the replacement for VirtueDesktops. This lead to Tony Arnold the main developer of VirtueDesktops downing tools and declaring VirtueDesktops dead.

So everybody waited in anticipation for OS X 10.5… only to find Spaces not living up to their expectations.

So why the big fuss?

With the advent of 10.5 Apple has tightened up the code such that no other Desktop managers properly work on top of Leopard. This leaves many followers of VirtueDesktops little or no choice than maybe to change their working practices to Spaces or find alternatives….

1. HyperSpaces

From the creator of VirtueDesktops comes HyperSpaces. Unfortunately this isn’t set for launch until 2008 and it also isn’t clear if it will have the same rich feature set that VirtueDesktops has.

2. YouControlDesktops

YouControlDesktops has probably the closest functionality you are going to get to VirtueDesktops. The interface for configuring things isn’t as intuitive or slick as Virtue but it’s not such a big deal since you’ll not need to use it that often.

Currently there is a beta version for 10.5 that works pretty well but there is some kinks. Namely issues with the Dock not always getting context when it should.

So in summary the solution is not perfect but it’s the best I’ve found so far. Download the beta and provide the developers with feedback, hopefully we can help push this forward.

Mephisto gets a core team

Mon, 22 Oct 2007 08:00:00 GMT

Good news for Mephisto users. The core team has been decided and the ideas for the 1.0 release are being finalised.

Interestingly enough there may also be a push to switch over to using Git which is gaining a bit of traction in the Rails community. I’m an avid user of SVK and gave Git a try a while back but stopped short after hearing tales of woe with git-svn… I may revisit at some point.

"script/generate undo" plugin released

Wed, 26 Sep 2007 14:23:00 GMT

Grab it while it’s hot:

 script/plugin install svn://rubyforge.org/var/svn/beyondthetype/undo_generator_plugin/trunk

Key Features

Undo any generator command
Keeps a log of all generator history in log/generator.log
Prompts before undo’ing any commands
Even works when you bail out of a generator command part way through