How to install rNOMADS with GRIB file support on Windows

Two years ago, I wrote a software package for R called “rNOMADS” that interfaces with online weather and sea ice model repositories to gather data in real time, for free.  The data are delivered in two ways: a simple, pure R, cross platform interface using GrADS-DODS, and binary files in GRIB format.  The one issue with GRIB is that this format can’t be read directly into R; it requires the external program “wgrib2.”  Installing rNOMADS with GRIB support for Linux is covered in this post (Mac OS is probably similar).  I thought GRIB support for Windows was impossible until the guy who runs this blog casually told me he’d figured it out.  So, I finally got around to trying it myself, and I’m happy to say I got it to work!  Here’s how:

Step 1:

Download the most recent wgrib2.exe version and all DLL files from Wesley Ebisuzaki’s web site:  Click here.

Step 2:

Make a directory somewhere on your computer.   I chose:  C:\Program Files\wgrib2

Copy the .exe and the .dll files to this directory.

Step 3:

Append the directory path to the Windows PATH variable.  Find out how to do this here.

Step 4:  If R is open, close and reopen it.  Then try the following command in the R interpreter:
system("wgrib2")

If you get a bunch of text that looks like the image below, you’ve succeeded.  You can start reading GRIB files with rNOMADS!

Output of wgrib2 call from R.

Output of wgrib2 call from R.

rNOMADS package has been published in the journal Computers & Geosciences

I’m happy to announce that my article “Near Real Time Weather and Ocean Model Data Access with rNOMADS” was recently accepted for publication in the journal Computers & Geosciences. The editor has kindly provided me with a link to the article that’s free until late April, check it out here.  I’m excited to have the package published somewhere official and I’m looking forward to seeing who ends up using it.

In other news, I’m still waiting for the tropospheric jet to point anywhere besides East so we can launch a tracked, camera bearing solar balloon from Chapel Hill.   I wrote an rNOMADS script that generates wind profiles from the 0600 GMT model run on today’s date out to three days from now, and put it on my computer’s crontab.  Every day, I look at the winds, and wait.  Here’s the profiles for today, let’s see if it updates daily on wordpress too:

Wind profiles above Chapel Hill, NC, USA

You can have a look at the script that generates this image here.

Note that the “montage” system command uses the Ubuntu imagemagick package to combine separate PNGs.  If you’re on windows or mac, this will not work for you.

New updates to the rNOMADS package and big changes in the GFS model

I rolled out a big update to the rNOMADS package in R about two weeks ago.  Now, the list of real time weather, ocean, and sea ice models available through rNOMADS updates automatically by scraping the NOMADS web site.  This way, changes in model inventories will be instantly reflected in rNOMADS without the need for a new version release.

Keep abreast of future updates to rNOMADS by subscribing to the mailing list here.  Feel free to ask for help or make comments on this list as well.

In other news, NOAA just updated the Global Forecast System to provide 0.25 x 0.25 degree output – doubling the resolution of the model!  Check out this crystal clear views of surface temperatures across the planet (source code below the image):

World temperature at 2 m above ground using the 0.25 x 0.25 degree output of the Global Forecast System model.

World temperature at 2 m above ground using the 0.25 x 0.25 degree output of the Global Forecast System model.

 


library(rNOMADS)
library(GEOmap)

#Get dates of model output
model.urls <- GetDODSDates(“gfs_0p25″)

#Find day of most recent model run
latest.model <- tail(model.urls$url, 1)

#Find most recent model run on that day
model.runs <- GetDODSModelRuns(latest.model)

#Get the most recent model (excluding analysis only)
latest.model.run <- tail(model.runs$model.run[which(grepl(“z$”, model.runs$model.run))], 1)

#Define model domain
time <- c(0,0) #Analysis model
lon <- c(0, 1439) #All longitude points
lat <- c(0, 720) #All latitude points
variables <- c(“tmp2m”) #Temperature 2 m above ground

#Get data from NOMADS real time server
tmp.data <- DODSGrab(latest.model, latest.model.run,
variables, time, lon, lat, display.url = FALSE)

#Reformat it
tmp.grid <- ModelGrid(tmp.data, c(0.25, 0.25), “latlon”)

#Define color scale
colormap <- rev(rainbow(500, start = 0 , end = 5/6))

#Plot it
image(x = tmp.grid$x, y = sort(tmp.grid$y), z = tmp.grid$z[1,1,,], col = colormap,
xlab = “Longitude”, ylab = “Latitude”,
main = paste(“World Temperature at Ground Level:”,
tmp.grid$fcst.date))

plotGEOmap(coastmap, border = “black”, add = TRUE,
MAPcol = NA)

Fall 2014 Solar Balloon Flights

We had a record number of successful solar balloon launches this fall:  a total of three!  Two of these balloons carried messages in bottles (in case they landed in water) and one just carried a handwritten note.  Unfortunately, no one has come across our messages as of now.  I suspect this means the bottles came down on land somewhere.  Probably some hunter will come across one two decades from now.

We’ve been using paint pigment to darken the balloons – it’s pretty labor intensive because you have to rub the powder into the plastic.  As a result, these balloons were not dark enough and so had pretty bad lift.  Our newest bag (currently under my desk in my graduate student office) is quite a bit darker, and I might give it a once over before I try and send it off.  It’ll be carrying a tracker and a camera, so we’re waiting on light winds before flying.

1.  The 7′ tetroon

Master solar balloon builder Mathew Lippincott sent me a 7′ tetroon to test out earlier last year.  I was pretty excited because I’ve never tried flying a tetroon.  We got it a little dark (but should have spent more time on it), and managed to get it to lift one bottle with a message inside.  Here it is orbiting a parking lot, bouncing off a tree, inching over a busy street at about 30′ elevation, and finally heading skyward:

Thankfully my friend didn’t film the street crossing, because I thought there was going to be a solar balloon/car collision for sure.  Students walking by were pointing and asking if it was a weather balloon.  Hardly!

2.  The 22′ tetroon fail

Not-so-master solar balloon builder glossarch (yours truly) tried to make a tetroon back in 2013.  I made a mistake somewhere down the line and it ended up looking like a giant pillowcase.  Nevertheless, I figured I could get it to fly…and I was right. A bystander thought it was a hang glider.  Come on!  A flying pillowcase holding a Trader Joe’s bag full of bottles resembles no hang glider I’ve ever seen.  To each their own, I guess…

The pillowcase being inflated using vacuum cleaner exhaust.

The pillowcase being inflated using vacuum cleaner exhaust.

3.  Halloween Solar Balloon!

What could be better than a paint dropcloth ghost hovering ominously over your town on Halloween?  I sealed the bag in record time (about 20 minutes) simply by unrolling a swath of paint dropcloth and ironing each edge together to make a cylinder.  Then we drew a scary ghost face on it, scrubbed some pigment on the plastic, and waited for Halloween.  Initially, we had a styrofoam tombstone as payload.  But the tombstone was too heavy, so we ended up just attaching a note and launching.  Because the bottom of the balloon was so poorly ballasted, the whole thing cavorted around in midair quite a bit, even turning sideways a couple of times.  Due to calm winds near the surface we had quite a few witnesses.

Inflating the beast.

Inflating the beast.

Haunting Chapel Hill on Halloween 2014.

Haunting Chapel Hill on Halloween 2014.

A big thanks to Xiao Yang for taking photos and video!  See his Flickr albums here.

Persistence of Vision Frisbee

The first time I saw a persistence of vision (POV) clock I thought: “Wouldn’t that be cool on the side of the frisbee?”. It is a fun weekend project to make one. Here is an action shot of the POV frisbee I made:

IMG_4897

It looks better in person – this picture is the best we could do with a Canon camera held on a level surface. The frisbee is spelling out ‘bovine aero’. Below is a picture of a test in a darkened room:

IMG_4863

This is the perfect project for Adafruit’s tiny, lightweight Arduino compatible Trinket. The total cost of the materials was around $30:

Assembly is as simple as hooking up an LED & resistor in series to each digital out pin, writing some code to blink the lights in the correct order (see below), and attaching the circuit to the frisbee. Below are some pictures taken during assembly:

IMG_20141013_142138647_HDRIMG_20141013_142156068_HDR

Some insights, if anyone else tries this:

  • I considered using a load sensor to determine the rotation rate of the frisbee (a = v^2/r). If you know the rotational speed of the frisbee, you can time the LED outputs to write more consistently spaced letters. However, it turns out the POV illusion looks OK if you use a fixed letter writing rate (I used an ‘on’ time 1 ms per vertical column of letter).
  • To mount the LEDs on the frisbee, I drilled holes in the side and covered them with electrical tape. Punching the LEDs through the tape provided a stable mount for the LEDS.
  • I used letters made up of 5×5 pixel blocks. A better POV display could use more vertically stacked LEDs, or even multicolored LEDs.

Click below to see the code I used:

int B[25] = {1,1,1,1,1, 1,0,1,0,1, 1,0,1,0,1, 0,1,0,1,0, 0,0,0,0,0};
int O[25] = {0,1,1,1,0, 1,0,0,0,1, 1,0,0,0,1, 0,1,1,1,0, 0,0,0,0,0};
int V[25] = {1,1,1,1,0, 0,0,0,1,0, 0,0,0,0,1, 0,0,0,1,0, 1,1,1,1,0};
int I[25] = {0,0,0,0,0, 1,0,0,0,1, 1,1,1,1,1, 1,0,0,0,1, 0,0,0,0,0};
int N[25] = {1,1,1,1,1, 0,1,0,0,0, 0,0,1,0,0, 0,0,0,1,0, 1,1,1,1,1};
int E[25] = {1,1,1,1,1, 1,0,1,0,1, 1,0,1,0,1, 1,0,1,0,1, 0,0,0,0,0};
int A[25] = {0,1,1,1,1, 1,0,1,0,0, 1,0,1,0,0, 0,1,1,1,1, 0,0,0,0,0};
int R[25] = {1,1,1,1,1, 1,0,1,0,0, 1,0,1,1,0, 0,1,0,1,1, 0,0,0,0,0};
int S[25] = {0,1,0,0,1, 1,0,1,0,1, 1,0,1,0,1, 1,0,0,1,0, 0,0,0,0,0};
int P[25] = {1,1,1,1,1, 1,0,1,0,0, 1,0,1,0,0, 0,1,0,0,0, 0,0,0,0,0};
int C[25] = {0,1,1,1,0, 1,0,0,0,1, 1,0,0,0,1, 1,0,0,0,1, 0,0,0,0,0};
int space[25] = {0,0,0,0,0, 0,0,0,0,0, 0,0,0,0,0, 0,0,0,0,0, 0,0,0,0,0};

int letter_time;
int write_time;

void setup() {
  //use pins 0-4 for output
  pinMode(0, OUTPUT);
  pinMode(1, OUTPUT);
  pinMode(2, OUTPUT);
  pinMode(3, OUTPUT);
  pinMode(4, OUTPUT);

  letter_time = 2; //delay between letters (ms)
  write_time = 1; //how long a led is activated for (ms)
}

//Write an letter to the POV display
void write_letter(int * letter) {
  //write letter, column by column
  for(int i = 0; i < 5; i++) {
    for(int j = 0; j < 5; j++) {
      digitalWrite(j, letter[j+i*5]);
    }
    delay(write_time);
  }

  //write space after letter
  for(int i = 0; i < 5; i++) {
    digitalWrite(i, 0);
  }
  delay(letter_time);
}

void loop() {
  write_letter(B);
  write_letter(O);
  write_letter(V);
  write_letter(I);
  write_letter(N);
  write_letter(E);
  write_letter(space);
  write_letter(A);
  write_letter(E);
  write_letter(R);
  write_letter(O);
  write_letter(space);
}

rNOMADS 2.0.2 released

I uploaded the newest version of rNOMADS to CRAN yesterday. This one has a new plotting function for wind altitude azimuth, and magnitude (see below for plot and source code). I also added a function for reading GRIB inventories, fixed a few typos, and tweaked a few functions. In other news, the rNOMADS subversion repository has migrated to R-Forge: http://rnomads.r-forge.r-project.org/.

I’ve also set up an rNOMADS mailing list; subscribe here if you need to ask for help or want to hear about new releases.

Wind profiles at each infrasound station in the Transportable Array (currently on the US East Coast). Ground surface is at the center of the plot, the top of the stratosphere is at the outer radius. Wind magnitudes are denoted by color, and azimuth by position around the circle.

Wind profiles at each infrasound station in the Transportable Array (currently on the US East Coast). Ground surface is at the center of the plot, the top of the stratosphere is at the outer radius. Wind magnitudes are denoted by color, and azimuth by position around the circle.


library(rNOMADS)

download.file("http://www.unc.edu/~haksaeng/rNOMADS/myTA.RDATA",
destfile = "myTA.RDATA")
load("myTA.RDATA")
#Find the latest Global Forecast System model run
model.urls <- GetDODSDates("gfs_0p50")
latest.model <- tail(model.urls$url, 1)
model.runs <- GetDODSModelRuns(latest.model)
latest.model.run <- tail(model.runs$model.run, 1)

#Get model nodes

lons <- seq(0, 359.5, by = 0.5)
lats <- seq(-90, 90, by = 0.5)
lon.ind <- which(lons <= max(myTA$lon + 360) & lons >= min(myTA$lon + 360)) - 1
lat.ind <- which(lats <= max(myTA$lat) & lats >= min(myTA$lat)) - 1
levels <- c(0, 46)
time <- c(0, 0)

#Get data
hgt.data <- DODSGrab(latest.model, latest.model.run, "hgtprs",
    time, c(min(lon.ind), max(lon.ind)), c(min(lat.ind), max(lat.ind)), levels)
ugrd.data <- DODSGrab(latest.model, latest.model.run, "ugrdprs",
    time, c(min(lon.ind), max(lon.ind)), c(min(lat.ind), max(lat.ind)), levels)
vgrd.data <- DODSGrab(latest.model, latest.model.run, "vgrdprs",
    time, c(min(lon.ind), max(lon.ind)), c(min(lat.ind), max(lat.ind)), levels)

#Reformat the data
hgt.grid <- ModelGrid(hgt.data, c(0.5, 0.5))
ugrd.grid <- ModelGrid(ugrd.data, c(0.5, 0.5))
vgrd.grid <- ModelGrid(vgrd.data, c(0.5, 0.5))

#Build profiles
zonal.wind <- c()
meridional.wind <- c()
height <- c()
for(k in 1:length(myTA$lon)) {
     hgt.profile <- BuildProfile(hgt.grid,
         myTA$lon[k], myTA$lat[k], spatial.average = TRUE)
    ugrd.profile <- BuildProfile(ugrd.grid,
        myTA$lon[k], myTA$lat[k], spatial.average = TRUE)
    vgrd.profile <- BuildProfile(vgrd.grid,
        myTA$lon[k], myTA$lat[k], spatial.average = TRUE)
    synth.hgt <- seq(min(hgt.profile),
        max(hgt.profile), length.out = 1000)
    ugrd.spline <- splinefun(hgt.profile, ugrd.profile, method = "natural")
    vgrd.spline <- splinefun(hgt.profile, vgrd.profile, method = "natural")
    zonal.wind[[k]] <- ugrd.spline(synth.hgt)
    meridional.wind[[k]] <- vgrd.spline(synth.hgt)
    height[[k]] <- synth.hgt
     print(paste("Finished", k, "of", length(myTA$lon), "profiles!"))
}

#Plot them all
PlotWindProfile(zonal.wind, meridional.wind, height, lines = TRUE, points = FALSE,
    elev.circles = c(0, 25000, 50000), elev.labels = c(0, 25, 50), radial.lines = seq(45, 360, by = 45),
    colorbar = TRUE, invert = FALSE, point.cex = 2, pch = 19, lty = 1, lwd = 1,
    height.range = c(0, 50000), colorbar.label = "Wind Speed (m/s)", elev.labels.az = 15)

A flight aboard the NASA High Altitude Student Platform (HASP)

The NASA High Altitude Student Payload (HASP) project provides a spot on a high altitude balloon payload for undergraduate and graduate students. When I heard about this last year, I gathered a team together, we applied and were accepted into the program. Our project: launch infrasound microphones into the stratosphere. Infrasound (sound at frequencies below audio range) is usually measured at the Earth’s surface, but we know it propagates hundreds of kilometers upward into the atmosphere. Our goal is to measure these sound waves as they cross the stratosphere.

The HASP project was definitely a commitment. As the team leader, I was required to write a monthly status report letting the HASP project leaders know what I was up to. I had to build my payload box under strict power draw, weight, and size limits. I also had to learn electronics from the ground up. Thankfully, another member of our team had lots of experience in electronics, so it wasn’t so bad.

Our Omnirecs DataCube logger installed in the payload box.

Our Omnirecs DataCube logger installed in the payload box.

A few weeks ago, I traveled to Palestine, Texas to bring my payload to the Columbia Scientific Balloon Facility (CSBF). There, our payload was subjected to extreme temperatures (ranging from -50 to 50 Celsius) and pressures (sea level to stratospheric).  We passed the test, recording the 8 Hz signal from the vacuum pump clearly even when the pressure was around 5% of sea level.  This was an important milestone: not only did it clear us for flight, but it also showed that our differential pressure microphones (constructed by Dr. Jeff Johnson at Boise State University) would operate in a near vacuum, something they were not designed to do.

Our payload about to face the thermal/vacuum test.

Our payload (the white box with the UNC logo) about to face the thermal/vacuum test.

From Texas, I traveled to New Mexico to launch our payload into the stratosphere.  I ended up staying in my home town, about 2.5 hours from the launch site at (a CSBF facility in Ft. Sumner, NM).  Needless to say, I spent a lot of time on the road!  I had to drive there and back three times: once to put my microphones on the flight ladder (see below), another time to make sure everything worked during the “hang test” (a dry run for launch), and finally for the big day itself – the flight.

CSBF staff mounting the infrasound microphones on the flight ladder.

CSBF staff mounting the infrasound microphones on the flight ladder.

Like any balloon flight, this one depended on the weather.  This time, the news was not good.  One group was ahead of HASP, and they had dibs on each launch window.  They tried twice, and were not able to fly both times.  I had to fly home on Saturday, so I showed CSBF and HASP personnel how to set up my payload, and I resigned myself to not seeing the balloon fly.

But as luck would have it, the previous team decided to wait, and a launch window opened Saturday morning, the day I was scheduled to fly home.  Since the flight was early in the morning and my plane ticket was for early afternoon, I decided I was going to go see the launch.  I drove out, arrived in Ft. Sumner at about 10 PM, slept in the back of the car for a few hours, then got up at 2:45 AM Saturday morning to start getting ready.

It's about 3:45 AM, and CSBF staff secure the microphones after I powered them up about 15 minutes before.  The wheel on "Big Bill" (the launch vehicle), is taller than I am.

It’s about 3:45 AM, and CSBF staff secure the microphones after I power them up. The wheel on “Big Bill” (the payload vehicle), is taller than I am.

The launch was touch and go the whole time – we had to wait for the winds to all blow in the same direction for the first 1000 ft in order to start the inflation process.  As luck would have it, they did straighten out, and the call was given to roll out and start inflating the balloon.

The balloon reaches full inflation about a half hour after sunrise.

The balloon reaches full inflation about a half hour after sunrise.

The launch was spectacular.  The balloon was released and drifted into the air.  Big Bill started driving in the direction that the balloon was going, and just when it was overhead, the payload was released.  The entire structure (800 ft high!) was now in free flight.  It seemed to climb slowly, but that was an illusion – when the balloon was 12,000 ft above the ground it still seemed close enough to touch.

The radio crackles and says "it's your balloon."  With that, the 3 million cubic ft envelope is released and begins climbing into the air.

The radio crackles and says “it’s your balloon.” With that, the 3 million cubic ft envelope is released and begins climbing into the air.

Here we go!

Here we go!

 

The balloon flew for about 8 hours, and was terminated over northeast Arizona.  Once the recovery team picks it up and ships us our data logger, we can find out what we heard up there.

A big thanks to the Louisiana State team for running HASP, and all the great people at CSBF who made it all happen!