Difference between revisions of "MAPREDUCE Elite"

Revision as of 13:48, 23 July 2015

#ENCODING
import io
import sys
sys.stdout = io.TextIOWrapper(sys.stdout.buffer, encoding='utf-16')
#MONGO
from pymongo import MongoClient
client = MongoClient()
client.progzoo.authenticate('scott','tiger')
db = client['progzoo']
#PRETTY
import pprint
pp = pprint.PrettyPrinter(indent=4)

Introducing the elite database **WORK IN PROGRESS

These questions will introduce the "elite" database, which contains data about the video game Elite Dangerous

There are two collections, commodities and systems. Inside systems there is are nested documents called stations
A system has many stations, and a station imports,exports, and bans many commodities

Document Structure

Unlike the previous examples, here we'll be using nested documents. A nested document is simply a document that contains other documents, for example:

[{"name":"Jim",
  "role":"Dad",
  "age": 42,
  "children":[{"name":"Bob", "age":2},
              {"name":"Alice", "age":3}
             ]
}]

Another change from the previous example is that null and empty values are no longer stored.
This means that some documents will have fields that others do not. Some systems will be uninhabited and have no stations. Some stations will have no listings.
To query null or non-existant fields we use <field>: null

The average commodities document looks something like this.

{
        "_id" : ObjectId("55af74e7402aa43f1ce7e3a3"),
        "name" : "Explosives",
        "average_price" : 267,
        "category" : "Chemicals"
}

systems is much bigger, so some stations and listings have been removed from this example

> db.systems.findOne({"name":"1 Kappa Cygni"})
{
        "_id" : ObjectId("55b0e227369fd571eca8764c"),
        "stations" : [
                {
                        "max_landing_pad_size" : "M",
                        "has_blackmarket" : 0,
                        "has_commodities" : 1,
                        "updated_at" : 1434929486,
                        "has_outfitting" : 0,
                        "government" : "Democracy",
                        "state" : null,
                        "has_shipyard" : 0,
                        "type" : "Unknown Outpost",
                        "has_rearm" : 0,
                        "allegiance" : "Federation",
                        "has_refuel" : 1,
                        "name" : "Kinsey Ring",
                        "listings" : [
                                {
                                        "commodity" : "Hydrogen Fuel",
                                        "supply" : 129630,
                                        "collected_at" : 1421669319,
                                        "update_count" : "1",
                                        "buy_price" : 93,
                                        "sell_price" : 89,
                                        "demand" : 0
                                },
                                {
                                        "commodity" : "Mineral Oil",
                                        "supply" : 0,
                                        "collected_at" : 1421669320,
                                        "update_count" : "1",
                                        "buy_price" : 0,
                                        "sell_price" : 326,
                                        "demand" : 217674
                                },
                        ],
                        "distance_to_star" : 2359,
                        "economies" : [
                                "Industrial",
                                "Refinery"
                        ],
                        "has_repair" : 1
                },
                {
                        "max_landing_pad_size" : "L",
                        "has_blackmarket" : 1,
                        "has_commodities" : 0,
                        "updated_at" : 1434929486,
                        "has_outfitting" : 1,
                        "government" : "Democracy",
                        "state" : null,
                        "has_shipyard" : 1,
                        "type" : "Unknown Starport",
                        "has_rearm" : 1,
                        "allegiance" : "Federation",
                        "has_refuel" : 1,
                        "name" : "Wohler Port",
                        "distance_to_star" : 3520,
                        "economies" : [
                                "Industrial",
                                "Refinery"
                        ],
                        "has_repair" : 1
                }
        ],
        "name" : "1 Kappa Cygni",
        "faction" : "United 1 Kappa Cygni Future",
        "government" : "Democracy",
        "allegiance" : "Federation",
        "updated_at" : 1430938622,
        "state" : "None",
        "needs_permit" : 0,
        "y" : 37.78125,
        "x" : -117.75,
        "security" : "High",
        "z" : 11.1875,
        "primary_economy" : "Industrial",
        "population" : 24843190
}

Here is a list of all the keys used.

    commodities: 
        _id, average_price, category, name
    systems: 
        _id, allegiance, faction, government, name, population, primary_economy, security, state, stations, updated_at, x, y, z

    systems.stations: 
        allegiance, distance_to_star, economies, export_commodities,has_blackmarket, has_commodities, has_rearm, has_repair,
        has_shipyard, has_outfitting, faction, government, listings, max_landing_pad, name, state, type, updated_at

    systems.stations.listings: 
        buy_price, collected_at, demand, commodity, sell_price, supply, update_count