Amazon SimpleDB
This section covers a simple way to load data from Amazon SimpleDB into Resolver One.
About Amazon SimpleDB
Amazon SimpleDB is a Web Service provided by Amazon. As you'd expect, it lets you store data on their servers as you would with any database. For full details about the service and how to use it, you should look at their website. However, here's the short version.
Amazon SimpleDB is a cut-down database system and structures data quite differently to a normal relational database—actually in a manner quite similar to the way a spreadsheet does the job.
Each database is structured into "domains", which can be regarded as similar to worksheets. A query to the database can only access a single domain (unlike relational database queries, which can "join" multiple tables). However, the domain is quite freely structured. It is made up of a number of items (analogous to worksheet rows), each item having a number of attributes (analogous to worksheet columns). There is no requirement for each row to have the same set of attributes
In relational database terms, this is highly denormalised! In Amazon's view, this makes it "...easy to use and provides the core functionality of a database - real-time lookup and simple querying of structured data - without the operational complexity."
Using Amazon's C# library
Amazon provides a C# library for accessing Amazon SimpleDB. Reusing this is much easier than writing your own web service access code to talk to the Amazon AWS systems directly.
Note
An independent group are currently working on a Python library.
Using the C# library as a .NET library from within Resolver One requires three steps...
1. Create the DLL
Here's how to create the required DLL:
- If you don't have a version of Visual C# installed, get the free "Express Edition" from Microsoft's website.
- Download the Amazon SimpleDB C# library from Amazon's website.
- Uncompress the C# library, and from Visual C# open the sample project amazon-simpledb-2007-11-07-cs-library\src\Amazon.SimpleDB.sln.
- Select Build, Build solution to compile the project.
- Take the file Amazon.SimpleDB.dll from amazon-simpledb-2007-11-07-cs-library\src\Amazon.SimpleDB\bin\Release and save it where you can use it later. (This DLL file contains the client access code for SimpleDB.)
2. Put data into your SimpleDB account
Before you try to load data from Amazon SimpleDB into your worksheet, you will probably want to add some data :-)
We recommend you...
Work through Amazon's Getting Started Guide, but skip the step "Deleting a Domain" at the end.
This leaves you with a domain called MyStore that holds some basic sample data.
(Alternatively, use a similar article in the IronPython Cookbook.)
3. Read the data from within a Resolver One spreadsheet
We've provided a sample RSL file that allows you to load up the contents of an AWS domain.
Procedure
Download this sample RSL file from the Resolver Exchange.
Save the file on your system.
Copy the .NET library to the same location.
Open the sample RSL file in Resolver One.
In the Settings worksheet, enter your AWS access key and secret keys. (If you want to access a domain other than the default "MyStore", you can change that here too.)
Let recalculation complete, then switch to the "Data" worksheet.
You should see all of the data from your domain. The Header Rows and Columns are set up so you can use them to query the data from your pre- or post-formula user code, or from your formulae.
About the code
Here's a step-by-step explanation of the pre-formulae user code that does the work.
Load the .NET library we created earlier...
import clr clr.AddReference("Amazon.SimpleDB") from Amazon.SimpleDB import AmazonSimpleDBClient from Amazon.SimpleDB.Model import GetAttributes, QueryConnect to the web service using the keys that were entered in the "Settings" worksheet (using references to named ranges)...
accessKeyId = accessKey.Value secretAccessKey = secretKey.Value service = AmazonSimpleDBClient(accessKeyId, secretAccessKey)
Set up the "Data" worksheet, giving it an appropriate header row and column...
sheet = workbook["Data"] itemHeader = "<Item>" rowNum = 1 colNum = 1 sheet.HeaderRow = sheet.Rows[rowNum] sheet.HeaderCol = sheet.Rows[colNum] sheet.HeaderRow[colNum] = itemHeader sheet.HeaderRow.Bold = True
Make a query to Amazon SimpleDB, asking for all items in the specified domain...
queryAction = Query().WithDomainName(domain.Value) response = service.Query(queryAction)
If the request returns any items, iterate over them...
if response.IsSetQueryResult(): for itemName in response.QueryResult.ItemName:Add each item to a new row, putting its name into the special "<Item>" column, then getting all of its attributes...
rowNum += 1 row = sheet.Rows[rowNum] row[itemHeader] = itemName domain = GetAttributes().WithDomainName("MyStore") getAttributesAction = domain.WithItemName(itemName) response = service.GetAttributes(getAttributesAction)If the item has any attributes, iterate over the attribute/value pairs that have been returned...
if response.IsSetGetAttributesResult(): for attribute in response.GetAttributesResult.Attribute:
The next step is to put the attributes into appropriate cells in the item's row. This is a little more complicated than the code so far.
Items in Amazon SimpleDB domains can specify a number of possible values for an attribute. For example, in the "Getting started" tutorial MyStore domain, the sweater's size can be "Large", "Medium" or "Small". The solution here is to use lists. In Resolver One, a cell can hold a list of values, so if an attribute for a specific row has multiple values, we can put a list of those values into the relevant cell.
Here's how the code does this...
If the current item already has an attribute with the name of the current attribute, changes the cell to use a list (if required) and adds the current value to the list. Otherwise puts the current value into the cell.
try: if row[attribute.Name] is Empty: row[attribute.Name] = attribute.Value else: if type(row[attribute.Name]) != list: row[attribute.Name] = [row[attribute.Name]] row[attribute.Name].append(attribute.Value) except IndexError: colNum += 1 sheet.HeaderRow[colNum] = attribute.Name row[attribute.Name] = attribute.Value
(The try...except that wraps around this section creates new columns for attributes as they are encountered for the first time.)
Comments
If you have comments, questions or suggestions about any of the Resolver One documentation, please post them to the Documentation Suggestions Forum.
