Accessing global weather model data using the rNOMADS package in R

*** UPDATE ***

rNOMADS 2.0.0 was released yesterday (5-15-2014), and it now has cross-platform support (meaning you can use its data retrieval capabilities in Windows and Mac as well as Linux).

Stay tuned for more tutorials and usage examples in the coming days.

***

The rNOMADS package interfaces with the NOAA Operational Model Archive and Distribution System to provide access to 60 operational (i.e. real time and prediction) models describing the state of the ocean and the atmosphere. rNOMADS has been used to get wind and wave data for a real time sailing game, to quantify solar energy available for power plants in Europe, and to predict helium balloon flights. We look forward to continuing to hear about novel and unexpected uses for this spectacular data set.

In this post, we show how to use rNOMADS to do the following:
1. Plot global temperature and wind at different atmospheric pressure levels
2. Produce a precise atmospheric temperature and wind profile for a specific point at a specific time

Links to source code are provided below each set of figures. A link to rNOMADS installation instructions is provided at the end of the post.

Global Temperature and Wind Maps

The Global Forecast System (GFS) model provides weather data on a 0.5 x 0.5 degree grid for the entire planet.  It is run 4 times daily, and produces a prediction every three hours out to 180 hours.  The zero hour “analysis” forecast is the state of the atmosphere at the model run time, and it uses some observational data to increase its accuracy.  Here, we plot the temperature at 2 m above the surface, the wind at 10 m above the surface, and the winds at 300 mb (around 9 kilometers elevation).  The 300 mb plot often shows the northern and southern jet streams quite well.

Temperature at the Earth's surface determined using the Global Forecast System model.

Temperature at the Earth’s surface determined using the Global Forecast System model.

Winds at the surface of the Earth from the GFS model.  Note the little spot of high winds - that's Tropical Cyclone Gillian, a Category 3 storm when this image was generated.

Winds at the surface of the Earth from the GFS model. Note the little spot of high winds south of Indonesia – that’s Tropical Cyclone Gillian, a Category 3 storm when this image was generated.

Jet streams and Rossby waves are clearly visible in this image of the wind speeds of the upper troposphere/lower stratosphere.

Jet streams and Rossby waves are clearly visible in this image of the wind speeds of the upper troposphere/lower stratosphere.

Download the source code for these images here.

 

Instantaneous Atmospheric Profile over Sakura-Jima volcano, Japan

It’s important to know which direction the winds are going directly above active volcanoes, because eruptions can carry ash into air space and over inhabited areas.  One impetus for the continued development of rNOMADS was to provide a one-stop solution for generating high precision wind profiles over specific points, allowing ash distribution prediction as soon as an eruption begins.  Here, we have generated a spatially and temporally interpolated wind and temperature profile over Sakura-Jima volcano, Japan.  The profile is calculated for the exact time when the source code is run.

Instantaneous temperature profile above Sakura-jima volcano, Japan.

Instantaneous temperature profile above Sakura-jima volcano, Japan.

An eruption at this instant would produce ashfalls east of the volcano for a plume height of 15 km.  However, if a truly massive 30 km high plume was produced, ashfalls would occur both east and west of the volcano.

An eruption at this instant would produce ashfalls east of the volcano for a plume height of 15 km. However, if a truly massive 30 km high plume was produced, ashfalls would occur both east and west of the volcano.

Download the source code for these images here.

 

Some of these scripts require the aqfig package in R to generate the colorbar legends.

Instructions on installing R, rNOMADS, and their dependencies are here.

Advertisements

Trajectory of the Qu8ke Rocket

Back in 2011, the Qu8ke rocket created quite a buzz with this amazing video from its flight:

One of the goals of this rocket was to win the Carmack Prize, an award for the first amateur rocket to obtains a GPS reading from over 100,000 ft. While this rocket did break the 100,000 ft mark, it unfortunately lost GPS lock during its high speed ascent. Analysis of the on board accelerometer data indicates that the rocket reached ~121,000 ft. In a previous post, we describe a method for determining the position a high altitude photograph was taken from using a set of landmarks, OpenCV, and information about the camera’s field of view. The video from the flight of the Qu8ke rocket provides an excellent opportunity to revisit this method and to confirm the accelerometer predicted elevation.

Step 1: Find the GoPro camera parameters

The Qu8ke rocket carried a GoPro camera, which recorded video of the flight in 1080p. To analyze this video with OpenCV, we first needed to find the camera matrix, which contains information about the field of view and radial distortion (fisheye) of the lens.  Following OpenCv’s  method for finding the camera matrix from a set of pictures with a known geometry (we used a chess board), we found that it was difficult to obtain a matrix that could reliably undistort an entire image taken by a GoPro. This may be due to the fact that OpenCV only uses three radial distortion coefficients, which are unable to fully describe the distortion in the image resulting from the 170 degree field of view lens. Despite this, the camera matrix is able to accurately describe mapping between points near the center of the image, as shown below.  We avoided this issue by only picking landmarks not close to the edges/corners of the image.

My beautiful picture

Original (left), Undistorted (Right)

Step 2: Extract a series of stills from Youtube video

We downloaded the source mp4 video from youtube with the 4K video downloader and extracted pictures using ffmpeg. For example the following command:

ffmpef -ss 100 -i qu8ke.mp4 -r 1 -qscale:v 2 %05.jpg

extracts relatively high quality jpegs from the movie every 1 s, starting 100 s into the video. ImageMagick’s convert tool was used to rotate, crop, and resize batches of images.

Step 3: Use OpenCV with a set of landmarks visible in the images to estimate the camera position:

Although the video from the flight is 1080p, the extracted still images aren’t very high quality due to motion blur and the melted plastic that accumulated on the lens (things get hot when you are going Mach 1+). A picture from the rocket near the apogee is shown below:

pnp_1The black circles are the human picked landmark locations, while the blue circles are the reprojected landmark locations given the predicted camera position and orientation. The fact that the circles overlap demonstrate that we have a self-consistent estimate of the camera pose. Adding small amounts of noise to the camera matrix and 2D/3D landmark locations, we are able to estimate the uncertainty in the camera pose. For most of the images we analyzed, the uncertainty in the position was below 6,000 ft. Using pictures throughout the flight like the one above, we reconstructed the trajectory of the rocket:

quake_flightOur predictions confirm that the Qu8ke rocket surpassed 100,000 ft in elevation, and indicate that the rocket went a bit higher (~13,000 ft) than predicted by the on board accelerometers. I am not an expert in interpreting accelerometer data, but I would guess that integrating the acceleration data over the course of the rocket flight becomes increasingly prone to error as time increases, which could account for the difference in the elevation predictions.

Balloon from Floral Arrangement Flies 135 Miles

Ever wondered how far a party balloon can fly if you let it go, and how long it stays in the air?  Turns out they can fly for over a hundred miles and for at least 8 hours or so.  I discovered this when I released an aluminized party balloon from the UNC Chapel Hill Campus with my email address attached.  This afternoon, I got an email from a resident of Pantego, North Carolina – a sleepy town of 200 people located right near the seashore, about 135 miles from Chapel Hill as the crow flies.  She found it in her front yard at about 10 AM today.

This is an example of the type of balloon I released. I think the inside’s coated with aluminum, as opposed to the stretchy ones you blow up with air.

I released it at about 8 AM yesterday morning, so it probably arrived sometime last night.  I’m actually surprised it travelled such a short distance given that it was in the air all day – but it could be that it did not take a direct flight path, stayed low to the ground (where winds are milder) or landed yesterday and she just didn’t see it.

She said that there were no rips in the balloon, so it must have lost its helium in transit.  This is rather surprising, since it sat in my lab for about 2 weeks and only lost about half its volume.  I assume that the increase in internal pressure compared to external pressure as the balloon rises increases the rate of helium loss.

The reason I had this balloon is because I recently passed an exam, so my mom sent me a floral arrangement with a balloon that said “Congratulations!  You did it!”  Initially I was curious to see how long it would take to lose its buoyancy.    Once it was clear that the end was near, I figured it was a pity to waste a perfectly good balloon, so I attached a message and let it go.

Launch: 8 AM, March 11th, at 35.908101 North, -79.052106 West

Landing: Sometime before 10 AM, March 12, at 35.589202 North, -76.662261 West

Balloon Disasters and Science Experiments Gone Wrong

Anyone who has ever tried to launch a balloon knows that a million things can go wrong (and will).  Anyone who has ever done science (that includes third grade science fair projects) knows that experiments often fail, sometimes utterly.  So when I had a chance to give a short talk on science gone wrong at the #ScienceFail event, I sure did not lack for subject material.  The talk is posted online here – I hope you get a few laughs at my expense!

I also discuss why I fly balloons.  But why do I have to even say it?  The pictures below are description enough.

A view from our still camera aboard our weather balloon.

A view from our still camera aboard our weather balloon.

View from the stratosphere.

View from the stratosphere.

Our first successful solar balloon lifts off.

Our first successful solar balloon lifts off.

A ten foot solar balloon ready to fly.

A ten foot solar balloon ready to fly.

 

Cellphone reception in the air measured with helium and solar balloons

The maximum height of cellphone reception varies quite a bit depending on where you are – we’ve seen 30,000 ft in New Mexico and 4,000 ft in North Carolina.  In this post, I’ll give a few examples of cellphone reception from altitude from our own experience.  I’ll also explain what factors probably led to the surprising range of maximum altitudes we observed.

Cell phones have been used to track high altitude balloons for a while now: at least since these guys from MIT launched a camera into near space for $150.  Here at Bovine Aerospace, we used cell phones for three launches: a near-space flight in New Mexico, a solar balloon near Norwood, Massachusetts, and a solar balloon near Chapel Hill, North Carolina.  We used the instamapper app, which went offline in 2012 but apparently is back (just realized this about 2 minutes ago, it’s like hearing back from an old friend).

The Jake 2 high altitude balloon launch carried a SPOT tracker and a Boost Mobile phone.  The phone continued to transmit latitude, longitude and altitude until the balloon reached 20,000 ft, after which the altitude ceiling was reached (some GPS chips stop sending altitude above a certain height).  It continued to report latitude and longitude to approximately 9,000 meters (30,000 ft), when the connection was lost.  The cell phone did not transmit data on the way down (good thing we had the SPOT!).  This is not too surprising because the balloon came down over a remote desert area and ended up in the mountains.

Unfortunately we do not have the cell phone data, because we never copied it down and the online repository expired.  Rookie mistake!

The Jake 3 solar balloon carried my coworker’s old Android phone to about 3,000 m (9,000 ft):

Elevation versus time plot for the Jake 3 solar balloon launch in Norwood, Massachusetts.

Elevation versus time plot for the Jake 3 solar balloon launch in Norwood, Massachusetts.

We had cellphone tracking from the surface (35 meters above sea level) to 350 m (1161 ft) above sea level.  We lost tracking for about twenty minutes, then regained it at 1867 m (6123 ft) above sea level.  Tracking was lost for good at 2755 m (9038 ft) above sea level, after about 43 minutes in the air.  The balloon was visible for 15 or 30 minutes after we lost tracking.  We’re not sure how high the balloon went, but we do know it came down in the Atlantic…7 of the 9 messages in bottles in the payload were discovered up and down the Massachusetts coastline.

The gap in coverage is particularly interesting since the balloon was never more than a mile from the launch site during tracking (despite being well over a mile in the air when contact was finally lost).  So, clearly there is not a simple relationship between cell reception and altitude.  I have a pretty good idea what is causing this – see the discussion at the end of the post.  You can download the position data for Jake 3 here.

We repeated this experiment with a different solar balloon design (and a different Android, obviously) in Chapel Hill, North Carolina a few months later.  Once we lost reception, we never expected to see the balloon again.  However, something went wrong during the flight and the balloon came back down a few miles away.  The resulting altitude plot looks like this:

Elevation versus time for the Jake 4 solar balloon flight in Chapel Hill, North Carolina.

Elevation versus time for the Jake 4 solar balloon flight in Chapel Hill, North Carolina.

Cellphone tracking was present from the surface (100 m, 328 ft) above sea level to 800 m (2624 ft) above sea level.  The balloon was too high for cellphone reception for about 21 minutes.  It reappeared, descending rapidly, at an elevation of 1164 m (3818 ft) above sea level.  We recovered the balloon, cell phone, and the messages in bottles we’d hoped to distribute at sea in a forest about 6 miles away.  Over a thousand foot difference in maximum height of cellphone reception over 6 miles!  Evidently, the maximum cellphone reception height is variable spatially as well.   You can download the position data for Jake 4 here.

While we still have no idea what made it come down suddenly, it is certainly possible to make a rough estimate of the maximum height of the balloon using linear regression.  Turns out it made it almost exactly a mile above sea level before something went wrong!

Maximum elevation reached during the Jake 4 flight estimated using linear regression.

Maximum elevation reached during the Jake 4 flight estimated using linear regression.

Evidently cellphone reception with altitude varies with location (highest in New Mexico, lowest in North Carolina), varies with horizontal distance in the same region (i. e. North Carolina), and can periodically disappear and reappear with altitude (Massachusetts).  All three of these variations make sense.  In New Mexico, cellphone networks must cover large areas of mountainous land with very low population density.  My guess is that cellphone transmitters in New Mexico send a very powerful signal in order to cover the largest land area with the fewest possible towers.  So it makes sense that we would still have cellphone reception above 25,000 ft in New Mexico.

The gap in cellphone transmission in Massachusetts was likely not due to the phone malfunctioning but actually a consequence of how radio antennas operate.  It turns out that directional antennas (such as those used to transmit cell phone signals along the ground) produce side lobes in addition to the intended radiation direction.  So this balloon remained in the main signal zone until about 2600 ft, where it entered a null region.  It then crossed into a side lobe at about 6100 ft, finally losing the signal again just over 9,000 ft above sea level.

Illustration of main and side lobes. Figure from the Wikipedia article on side lobes in radio antenna design.

As an aside, this same radiation pattern shows up in some unexpected places – such as the sound pattern produced by speakers.

The North Carolina launch showed something a bit more mundane – cell phone reception varies with location!