Cookies help us deliver our services. By using our services, you agree to our use of cookies. More information

Difference between revisions of "MAPREDUCE Elite"

From NoSQLZoo
Jump to: navigation, search
Line 39: Line 39:
 
<div class=q data-lang="py3">The <code>commodities</code> collection contains the <code>name</code> and <code>average_price</code> of each commodity.<br/>
 
<div class=q data-lang="py3">The <code>commodities</code> collection contains the <code>name</code> and <code>average_price</code> of each commodity.<br/>
 
There are 99 unique commodities and 15 categories.
 
There are 99 unique commodities and 15 categories.
<p class="strong>Find the average price of each category, round to the nearest whole number</p>
+
<p class="strong">Find the average price of each category, round to the nearest whole number</p>
 
<pre class=def>
 
<pre class=def>
 
pp.pprint(
 
pp.pprint(
Line 47: Line 47:
 
<div class="ans">
 
<div class="ans">
 
from bson.code import Code;temp = db.commodities.map_reduce( map=Code("function(){emit(this.category,this.average_price)}"), reduce=Code("""function(key,values){var total = 0;for (var i = 0; i < values.length; i++){total += values[i];}return Math.round(total/values.length);} """),out={"inline":1} );pp.pprint(temp['results'])
 
from bson.code import Code;temp = db.commodities.map_reduce( map=Code("function(){emit(this.category,this.average_price)}"), reduce=Code("""function(key,values){var total = 0;for (var i = 0; i < values.length; i++){total += values[i];}return Math.round(total/values.length);} """),out={"inline":1} );pp.pprint(temp['results'])
 +
</div>
 +
</div>
 +
 +
<div class=q data-lang="py3">
 +
<p class="strong">There are many types of Government. Show the amount of people for each type.</p>
 +
<div class="hint" title="Assertion errors, dealing with non-existent values">
 +
It's not possible to <code>emit</code> on a field that doesn't exist. Use <code>query</code> to only include documents where the fields you need exist.
 +
</div>
 +
<pre class=def>
 +
</pre>
 +
<div class="ans">
 +
temp = db.systems.map_reduce( query={"government":{"$exists":1},"population":{"$exists":1}}, map=Code("function(){emit(this.government,this.population)}"), reduce=Code("""function(key,values){var total = 0;values.forEach(function(value){total += value;});return total;} """), out={"inline":1} );pp.pprint(temp['results'])
 
</div>
 
</div>
 
</div>
 
</div>
Line 53: Line 65:
 
<b>The Alliance</b> consists of independent systems, though an independent system is not necessarily part of <b>The Alliance</b>. In such a case their allegiance will be stored as "<b>Independent</b>"<br/>
 
<b>The Alliance</b> consists of independent systems, though an independent system is not necessarily part of <b>The Alliance</b>. In such a case their allegiance will be stored as "<b>Independent</b>"<br/>
 
There are a few systems that come under <b>Anarchy</b>. Non-populated systems without stations do not have an allegiance, and should be ignored.  
 
There are a few systems that come under <b>Anarchy</b>. Non-populated systems without stations do not have an allegiance, and should be ignored.  
<div class="hint" title="Dealing with None">
 
It's not possible to <code>emit</code> on a field that doesn't exist. Use <code>query</code> to remove documents where <b>allegiance</b> doesn't exist.
 
</div>
 
 
<p class="strong>Show the amount of systems following each type of allegiance.</p>
 
<p class="strong>Show the amount of systems following each type of allegiance.</p>
 
<pre class=def>
 
<pre class=def>
Line 62: Line 71:
 
temp = db.systems.map_reduce( query={"allegiance": {"$ne":None}}, map=Code("function(){emit(this.allegiance, 1)}"), reduce=Code("""function(key,values){var total = 0;values.forEach(function(value){total += value;});return total;} """), out={"inline":1} );pp.pprint(temp['results'])   
 
temp = db.systems.map_reduce( query={"allegiance": {"$ne":None}}, map=Code("function(){emit(this.allegiance, 1)}"), reduce=Code("""function(key,values){var total = 0;values.forEach(function(value){total += value;});return total;} """), out={"inline":1} );pp.pprint(temp['results'])   
 
</div>
 
</div>
</div>
 
 
<div class=q data-lang="py3">
 
<p class="strong></p>
 
<pre class=def>
 
</pre>
 
 
</div>
 
</div>
  
Line 77: Line 80:
 
&nbsp;&nbsp;&nbsp;&nbsp;3 are allied to <b>The Federation</b><br/>
 
&nbsp;&nbsp;&nbsp;&nbsp;3 are allied to <b>The Federation</b><br/>
 
&nbsp;&nbsp;&nbsp;&nbsp;5 systems are <b>Independent</b><br/><br/>
 
&nbsp;&nbsp;&nbsp;&nbsp;5 systems are <b>Independent</b><br/><br/>
<p class="strong>Show the allegiance of each of the Power's systems.</p>
+
<p class="strong">Show the allegiance of each of the Power's systems.</p>
 
<pre class=def>
 
<pre class=def>
 
</pre>
 
</pre>

Revision as of 16:50, 24 July 2015

#ENCODING
import io
import sys
sys.stdout = io.TextIOWrapper(sys.stdout.buffer, encoding='utf-16')
#MONGO
from pymongo import MongoClient
client = MongoClient()
client.elite.authenticate('scott','tiger')
db = client['elite']
#PRETTY
import pprint
pp = pprint.PrettyPrinter(indent=4)
#JS
from bson.code import Code

Introducing the elite database **WORK IN PROGRESS

These questions will introduce the "elite" database, which contains data about the video game Elite Dangerous

There are two collections, commodities and systems. Inside systems there is are nested documents called stations
A system has many stations, and a station has many trade listings

Keys used in this database.

    commodities: 
        _id, average_price, category, name
    systems: 
        _id, allegiance, faction, government, name, population, primary_economy, security, state, stations, updated_at, x, y, z

    systems.stations: 
        allegiance, distance_to_star, economies, export_commodities,has_blackmarket, has_commodities, has_rearm, has_repair,
        has_shipyard, has_outfitting, faction, government, listings, max_landing_pad, name, state, type, updated_at

    systems.stations.listings: 
        buy_price, collected_at, demand, commodity, sell_price, supply, update_count
        

Read more about the structure here: Elite Document Structure

Questions

The commodities collection contains the name and average_price of each commodity.

There are 99 unique commodities and 15 categories.

Find the average price of each category, round to the nearest whole number

pp.pprint(
    db.commodities.find_one()
)

from bson.code import Code;temp = db.commodities.map_reduce( map=Code("function(){emit(this.category,this.average_price)}"), reduce=Code("""function(key,values){var total = 0;for (var i = 0; i < values.length; i++){total += values[i];}return Math.round(total/values.length);} """),out={"inline":1} );pp.pprint(temp['results'])

There are many types of Government. Show the amount of people for each type.

It's not possible to emit on a field that doesn't exist. Use query to only include documents where the fields you need exist.


temp = db.systems.map_reduce( query={"government":{"$exists":1},"population":{"$exists":1}}, map=Code("function(){emit(this.government,this.population)}"), reduce=Code("""function(key,values){var total = 0;values.forEach(function(value){total += value;});return total;} """), out={"inline":1} );pp.pprint(temp['results'])

Each system has an allegiance. There are three main factions: The Federation, The Empire, and The Alliance.

The Alliance consists of independent systems, though an independent system is not necessarily part of The Alliance. In such a case their allegiance will be stored as "Independent"
There are a few systems that come under Anarchy. Non-populated systems without stations do not have an allegiance, and should be ignored.

Show the amount of systems following each type of allegiance.


temp = db.systems.map_reduce( query={"allegiance": {"$ne":None}}, map=Code("function(){emit(this.allegiance, 1)}"), reduce=Code("""function(key,values){var total = 0;values.forEach(function(value){total += value;});return total;} """), out={"inline":1} );pp.pprint(temp['results'])

Harder Questions

A power_control_faction or Power is an individual or organisation who is in control of a system.

These powers have allegiances, but the systems they control do not nescessarily have the same allegiance as they do.

Example: At the time of writing Zemina Torval is allied with the Empire and controls 47 systems.
    39 are allied to The Empire
    3 are allied to The Federation
    5 systems are Independent

Show the allegiance of each of the Power's systems.