Google Earth Community: Dynamic Air Quality and China...Mexico added

I have been working toward creating a dynamic global air quality layer that updates hourly on Google Earth. The effort is done as an outsider, and uses web crawling to return data from any agency that chooses to publish their data on a web page.

China has posed a unique exercise, and that it is running "again", I thought to share the effort.

The Chinese list air quality through their State Environmental Protection Administration . Their data including the city name, air pollution index and pollutant is only available in Chinese. (The English version shows the pollution index, but not the pollutant -- and a more complete version was shut down a month ago). That the cities are in Chinese characters precludes even the slightest clue as to how to interpret the data. For example, check out a air data page:

Chinese Version of Air Data

So I had the idea to use Google to automatically translate the page, and then scrape the translated page. The Chinese have three summary pages that are the basis for this content layer, and we can view the same page now translated.

Translated Chinese Air Quality Data

Now the next question is where are the Chinese cities? This posed its own challenge. Google Earth or Maps did not seem to know these locations. In this case I found (like many others probably know), that Wikipedia has a great geographic database of place names. Their city naming seems to align with the translated city naming from Google, but I did see many flavors of spelling of Chinese cities.

So now the data from the Chinese is regularly scraped, then scrubbed (as the translated web page has lots of cleanup requirements), and then joined with the city location database.

The Chinese data has other interesting challenges that reveal themselves. Each government can characterize their air quality at different interpretations of health effects. The Chinese do not use the same characterization as other countries. A value judgment for the map maker exists whether one is to be respectful of a countries rating, or seek to find a global norm. Now the map maker enters the greater politic.

Still I am pleased that this is up and running again. This layer had to be updated based on the Chinese stopping their English version. I have my concerns as to how long this dynamic scrape will be stable.

Hope you enjoy it. (Recall that it has 4 countries now, so spin the globe to China...it has so many huge cities I learned about through this exercise.)

...added Mexico over the weekend, and replaced the generic icon with a unique air quality icon.

Edited by bob_wenzlau (03/03/08 08:46 AM)