Tuesday, July 17

Tutorial: Embed eBird Rarities Data on your Website

If you're a web-developer, programmer, website owner, or blogger, you may wish to embed eBird Data on your website. What are the advantages?
  • Fast download time, much faster than a Google Gadget
  • Flexible design, so you can customize it so it blends into your website
  • You can write your code in JavaScript, an easy-to-learn and easy-to-use programming language
In this post I'll walk you through a code designed to display recent rarities in your area. All you need is a little tech-savviness, a website, and an hour or two.

Goal:
Using eBird APIs, a set of XML databases containing recent bird observations, we'll output the latest rarities in a region and give it a little formatting. In the end, your result should look something like this:

Developer Note: We won't be able to take advantage of the XHR access-control-allow-origin:* header, which permits cross-domain client-side XHR for the website in which it was indicated. eBird has not yet implemented this header. We will therefore depend either on APIs or platform-dependent cross-domain capabilities, or manually request permission for cross-domain access.
Step 1: What API do you want to use? To find out what an eBird API XML file looks like, click here. This example shows rarities were found in New York within the last 7 days. The URL of this API is:

http://ebird.org/ws1.1/data/notable/region/recent?rtype=subnational1&r=US-NY&back=7&detail=full

  • The first part (http://ebird.org/ws1.1/data/notable/region/recent) is the root URL which specifies an API which outputs rarities in a subnational region (e.g. state, province, county).
Developer Note: A different API root URL may be advantageous to your specific purposes. Check out the eBird APIs website for a full listing of your options.
  • The remainder of the URL defines parameters:
    • rtype=subnational1 specifies whether we would like to refer to a state/province (subnational1) or a county (subnational2). Change this depending on what region type you would like to use.
    • r specifies the subnational region name/ID.
           If you wish to enter a state/province, enter the country code, a hyphen, and the state code with no spaces (e.g. Montana = US-MT, North Dakota = US-ND, British Columbia = CA-BC, Noord-Holland Netherlands = NL-NH). If you do not know what state code to enter, click here, search for your state code, and copy over the "subnational-1" value of that line.
           If you wish to enter a county, click here and change the country code in the web address URL bar if you are not looking for a county within the United States. Find the county with the name you are looking for and the corresponding state code (subnational1 value) in the same line. Copy the subnational-2 code into your API URL. An example of a subnational2-code might be US-NY-109 for Tompkins County, New York.
    • back specifies how many days back you would like to look. You can only look back a maximum of 30 days. Change this accordingly
    • detail specifies how much detail you would like to see in the API. Putting in detail=full rather than detail=simple returns items such as observer names and checklist IDs, which we definitely don't want to miss! Type in detail=full, you don't stand a thing to lose.

Step 2: The HTML Code
The code you will write in this tutorial contains a very short HTML snippet:
<table id="raritiesContent"><tr><td>&nbsp;</td></tr></table>
You need to copy this code to the place in your website HTML document that you want to post this script. Paste the above code at that location.

The Javascript code you will begin too write in the next step will dynamically input the API content into this table element.

Step 3: Begin the JavaScript Code - XMLHttpRequest
Now that you've figured out what API you're going to use, and have put your HTML into it's place, we're going to write a JavaScript code that downloads the eBird API, parses it, and inserts the finished product into the <table> element. This javascript should be included in the <head> section of your webpage, or in a special section denoted by <script type="text/javascript"> script </script>.

The XMLHttpRequest, or XHR, tells the browser to download the XML file. It does so using the following code--copy it down into your new JavaScript file/header/inline script section:
var url = "http://ebird.org/ws1.1/data/notable/region/recent?rtype=subnational1&r=US-NY&back=7&detail=full"; // modify URL to whatever API you chose

// Request XML file -- standard XMLHttpRequest methods
var xhr = new XMLHttpRequest();
xhr.open("GET",url,false);
xhr.send();
var response=xhr.responseXML;
Step 4: Retrieve Data from the API
Now that we have downloaded our API, we can retrieve data and store it as variables which we will use later on. The following code will also define an empty string, "html", to which we will append all of our content. This content will eventually be pushed into the HTML code we have dealt with in step 2. Copy the following code verbatim into your JavaScript:
var sighting=response.getElementsByTagName("sighting"); // establish "bookmarks" at sighting tags, so that we can gather information for each sighting.

var html=""; // define empty string so that we can append tidbits later on inside the "for" loop

// begin loop which will sequentially traverse all sightings tags, so that we can gather data about each observation
for (i=0;i<sighting.length;i++) {

    // begin by declaring and editing variables
    var d=getRaritiesData(sighting.item(i)); // use the getRaritiesData function (discussed later) to extract the value of each noted tag in the API document
    if(d["how-many"]>0){ // if species count is a number, use a number
        var count=d["how-many"];
    }
    else{ // if species count is undefined, output X
        var count="X";
    }
    var valid = d["obs-valid"]; // is species reviewed and approved?
    var name = d["com-name"]; // species common name
    var date = d["obs-dt"]; // observation date. This is a messy string so we will edit it later
    var location = d["loc-name"]; // location name
    var locationPrivate = d["location-private"]; // is location private, if not it must be a hotspot
    if(d["first-name"]){ // if the observer is specified in the API document, output the first and last names
        var observer = d["first-name"] + " " + d["last-name"];
    }
    else{ // if observer is not specified in API document, output "Anonymous Observer"
        var observer = "Anonymous Observer";
    }
    var county = d["subnational2-name"]; // county name
    var state = d["subnational1-code"].substring(3,5); // state/province postal code
    var latitude = d["lat"]; // latitude
    var longitude = d["lng"]; // longitude
    var checklist = d["sub-id"]; // checklist identification number
    var locationColor = "black"; // set the default location color to black
    if(locationPrivate=="false"){locationColor="red";} // set location color to red if it is a hotspot
    var datestr = date.substring(5,10); // begin cleaning up the date variable and sort it into a month, a day, and a time of day.
    time = date.substring(11,16); // this is the time of day (e.g. 13:27)
    if(time.substring(0,1)==0){ // if the time of day begins with a 0, (e.g. 08:34), then take off the zero
        time=time.substring(1,8)
    }
    var month = datestr.substring(0,2)*10/10;
    var day = datestr.substring(3,5)*10/10;
Step 5: Append Data to "html" String
Now it's time to append our data to our "html" string. Using the variables defined above, we'll now put them into the string containing the HTML formatting and layout. This is the bit of code to modify in order to make the end result look different. Notice that we are still inside the "for" loop initiated in the previous step, so we create a new <tr> element for every sighting. Copy this code beneath the code you pasted above:
    // create a single row for the observation and begin appending data to "html" string
    html+="<tr><td>";
    html+=count + "<br />"; // in the first cell put a count...
    if(valid=="true"){ // ... and if observation is approved place a checklist icon
        html+='<img src="https://lh3.googleusercontent.com/-YNCzIb-YYMM/T8ksBMTlCZI/AAAAAAAAAqg/gGPdl23BdMc/s24/checkmark.png" style="height:16px;" />';
    }
    else if(valid=="false"){ // ... or if it has not yet been reviewed leave it empty.
        html+=' ';
    }
    html += "</td><td>";
    html += "<b>" + name + "</b><br />" + observer; // in this cell put species name and observer
    html += "</td><td>";
    html += "<td>" + month + "/" + day + "<br />" + time; // in this cell put the month/day and time that was calculated in step 4
    html += "</td><td>";
    html += "<span style='color:" + locationColor + ";'>" + location + "</span>;" // in this cell put location with the appropriate color generated above to show if it is a hotspot or not...
    html += "<br />" + county + " County, " + state + " "; // ... and the county followed by the state/province postal abbreviation...
    html += "(<a target='_blank' href='http://maps.google.com/?ie=UTF8&t=p&z=13&q=" + latitude + "," + longitude + "&ll=" + latitude + "," + longitude + " st'>map</a>)"; //... and end with a (map) linking to Google Maps with the location's GPS coordinates
    html += "</td><td>";
    html += "<a href='http://ebird.org/ebird/view/checklist?subID=" + checklist + "'>Checklist</a>"; // in the last cell give a link to the eBird checklist
    html += "</td></tr>"; // end the row for this observation
}
Step 6: Function to Get Data for each Sighting
Earlier we used the function getRaritiesData() to magically retrieve the ith sighting tag data. Here's the function that does that. Copy it beneath your other codes:
function getRaritiesData(n) { // returns the tag value for the tag name "n". Note that n was passed into the function as sighting.item(i)
    var d = new Object();
    var nodeList = n.childNodes;
    for (var j = 0; j < nodeList.length ; j++) {
        var node = nodeList.item(j);
        d[node.nodeName] = node.firstChild.nodeValue;
    }
    return d; // return tag value
}
Step 7: Output the Code
We've generated all the code to go inside our HTML <table> element, we just need to stick it in now. Add this code to the end of your javascript:
document.getElementById("raritiesContent").innerHTML = html;
Step 8: Permissions and Cross-domain XMLHttpRequest Policies
If you are using Blogger, Chrome platforms, other Google code products, web apps, or extensions, you can skip this step. Instead, research whether or not you will be able to access eBird's APIs without requesting permissions or cross-domain exemptions--remember that security measures are taken for cross-domain XMLHttpRequests, and you won't just be able to access eBird's APIs from your own domain.
     If you are hosting your own site, or are on a platform that does not provide open cross-domain access to any website, you will need to request access to eBird.org. To get access to eBird's APIs from your domain, send an email to ornith-is-admin@cornell.edu indicating you would like to be added to their http://ebird.org/crossdomain.xml file only after you have finished writing the code and are sure your website does not work using standard cross-domain XHR. eBird may implement the access-control-allow-origin:* header in the near future, so watch for this. If you have doubts about this, just email support@birdventurebirding.com anytime. You should receive a personal confirmation email from eBird verifying that you have been added to the crossdomain.xml file, thereby giving you developer privileges to access any page in the site via XHR.

Step 9: Finishing, Debugging, and Styling
After you have finished pasting in the codes above in their appropriate positions, it's time to see if it works. Check out your webpage. After three seconds of agony/anticipation as your page loads, your script will either work or not work. If it doesn't work, I recommend debugging it with Google Chrome browser Error Console. Simply go to options > tools > JavaScript console. If you are more novice, I recommend downloading a basic JavaScript editor like Yaldex which can easily pinpoint basic syntax errors.

If worse comes to worst, I'm a great debugger as well!! If you have issues, or a debugging error to ask me about, I'm here to answer any questions. Email support@birdventurebirding.com.

A Recap: The Code in One Chunk
Here's the code all in one chunk:
var url = "http://ebird.org/ws1.1/data/notable/region/recent?rtype=subnational1&r=US-NY&back=7&detail=full"; // modify URL to whatever API you chose

// Request XML file -- standard XMLHttpRequest methods
var xhr = new XMLHttpRequest();
xhr.open("GET",url,false);
xhr.send();
var response=xhr.responseXML;

var sighting=response.getElementsByTagName("sighting"); // establish "bookmarks" at sighting tags, so that we can gather information for each sighting.

var html=""; // define empty string so that we can append tidbits later on inside the "for" loop

// begin loop which will sequentially traverse all sightings tags, so that we can gather data about each observation
for (i=0;i<sighting.length;i++) {

    // begin by declaring and editing variables
    var d=getRaritiesData(sighting.item(i)); // use the getRaritiesData function (discussed later) to extract the value of each noted tag in the API document
    if(d["how-many"]>0){ // if species count is a number, use a number
        var count=d["how-many"];
    }
    else{ // if species count is undefined, output X
        var count="X";
    }
    var valid = d["obs-valid"]; // is species reviewed and approved?
    var name = d["com-name"]; // species common name
    var date = d["obs-dt"]; // observation date. This is a messy string so we will edit it later
    var location = d["loc-name"]; // location name
    var locationPrivate = d["location-private"]; // is location private, if not it must be a hotspot
    if(d["first-name"]){ // if the observer is specified in the API document, output the first and last names
        var observer = d["first-name"] + " " + d["last-name"];
    }
    else{ // if observer is not specified in API document, output "Anonymous Observer"
        var observer = "Anonymous Observer";
    }
    var county = d["subnational2-name"]; // county name
    var state = d["subnational1-code"].substring(3,5); // state/province postal code
    var latitude = d["lat"]; // latitude
    var longitude = d["lng"]; // longitude
    var checklist = d["sub-id"]; // checklist identification number
    var locationColor = "black"; // set the default location color to black
    if(locationPrivate=="false"){locationColor="red";} // set location color to red if it is a hotspot
    var datestr = date.substring(5,10); // begin cleaning up the date variable and sort it into a month, a day, and a time of day.
    time = date.substring(11,16); // this is the time of day (e.g. 13:27)
    if(time.substring(0,1)==0){ // if the time of day begins with a 0, (e.g. 08:34), then take off the zero
        time=time.substring(1,8)
    }
    var month = datestr.substring(0,2)*10/10;
    var day = datestr.substring(3,5)*10/10;


    // create a single row for the observation and begin appending data to "html" string
    html+="<tr><td>";
    html+=count + "<br />"; // in the first cell put a count...
    if(valid=="true"){ // ... and if observation is approved place a checklist icon
        html+='<img src="https://lh3.googleusercontent.com/-YNCzIb-YYMM/T8ksBMTlCZI/AAAAAAAAAqg/gGPdl23BdMc/s24/checkmark.png" style="height:16px;" />';
    }
    else if(valid=="false"){ // ... or if it has not yet been reviewed leave it empty.
        html+=' ';
    }
    html += "</td><td>";
    html += "<b>" + name + "</b><br />" + observer; // in this cell put species name and observer
    html += "</td><td>";
    html += "<td>" + month + "/" + day + "<br />" + time; // in this cell put the month/day and time that was calculated in step 4
    html += "</td><td>";
    html += "<span style='color:" + locationColor + ";'>" + location + "</span>;" // in this cell put location with the appropriate color generated above to show if it is a hotspot or not...
    html += "<br />" + county + " County, " + state + " "; // ... and the county followed by the state/province postal abbreviation...
    html += "(<a target='_blank' href='http://maps.google.com/?ie=UTF8&t=p&z=13&q=" + latitude + "," + longitude + "&ll=" + latitude + "," + longitude + " st'>map</a>)"; //... and end with a (map) linking to Google Maps with the location's GPS coordinates
    html += "</td><td>";
    html += "<a href='http://ebird.org/ebird/view/checklist?subID=" + checklist + "'>Checklist</a>"; // in the last cell give a link to the eBird checklist
    html += "</td></tr>"; // end the row for this observation
}

function getRaritiesData(n) { // returns the tag value for the tag name "n". Note that n was passed into the function as sighting.item(i)
    var d = new Object();
    var nodeList = n.childNodes;
    for (var j = 0; j < nodeList.length ; j++) {
        var node = nodeList.item(j);
        d[node.nodeName] = node.firstChild.nodeValue;
    }
    return d; // return tag value
}

document.getElementById("raritiesContent").innerHTML = html;

I have tried the above code in standard webpages on all major browsers, and it worked. I could not get this version to work on Google blogs (this website included), so unfortunately I can't provide an on-site demo.

If you have gone through this entire tutorial, and done a little debugging, and your code still doesn't work, feel free to email me at support@birdventurebirding.com. Provide as many details as you can. Also comment if you have been successful and would like to share your finished product with the rest of us. We love success stories! Overall, I hope this technology can be used on many webpages to help users get a more friendly recent sightings report. This is the best alternative to Google Gadgets, and I hope many of you make efforts to implement this code. Go eBird!

No comments:

Post a Comment

Please leave your comment below. If you have a question or issue which may be specific to your situation only, please email support@birdventurebirding.com. Thank you!