CSCE-315: Programming Studio (Summer 2012)

Project 1: Database Management System

Due dates and updates

Here are the various due dates. See near the end for details (i.e., what you need to submit for each submission window.

  1. Design documents (due 6/1 Friday)
  2. Parser code (due 6/5 Tuesday)
  3. DBMS engine code (due 6/8 Friday)
  4. Final project code + report (due 6/11 Monday)

Any updated info about the project will also be posted here.

Team configuration

This project is a team project, ideal team size is three. If the number of students is not divisible by three, we can have a few teams with size close to three. The teams will be assigned by the instructor, based on the programming proficiency survey.

In a nutshell

This is a two-part project. In the first part, the task is a to implement a simple database management system (DBMS). In the second part, the task is to implement an example application that relies on the DBMS system developed in the first part for its data manipulation needs.

Specification of the DBMS

Database management systems are very complex pieces of software. They support concurrent use of a database, transactions, permission handling, query optimizations, logging, you name it. To be efficient, they utilize highly tuned algorithms developed during the course of decades. So obviously, for a two-week long project, we have to simplify a lot. We thus base our DBMS on relational algebra.

Relational algebra is a formal system for manipulating relations. It consists of only six primitive operations. Each of the operations take relations as arguments, and produce a relation as a result. The operations thus compose freely.

The upside of using relational algebra is that the implementation effort of the DBMS stays manageable. The downside is that queries tend to be more verbose and maybe a bit harder to construct than, say, with SQL.

Terminology:

Database
a collection of relations
Relation
a table with columns and rows
Attribute
a named column of a relation
Domain
the set of admissible values for one or more attributes
Tuple
a row of a relation (sequence of values, one for each attribute of a relation)

Relational algebra

The six operations of (the core of) relational algebra are:

  1. Selection: select the tuples in a relation that satisfy a particular condition.
  2. Projection: select a subset of the attributes in a relation.
  3. Renaming: rename the attributes in a relation. OPTIONAL
  4. Set union: compute the union of two relations; the relations must be union-compatible. OPTIONAL
  5. Set difference: compute the set difference of two relations; the relations must be union-compatible. OPTIONAL
  6. Cross product: compute the Cartesian product of two relations.

Grammar

The communication with the DBMS takes place using a domain-specific language. The grammar of queries in this language is as follows.

query ::= relation-name <- expr ;

relation-name ::= identifier

identifier ::= alpha { ( alpha | digit ) }

alpha ::= a || z | A || Z | _

digit ::= 0 || 9

expr ::= atomic-expr
             | selection
             | projection
             | renaming
             | union
             | difference
             | product

atomic-expr ::= relation-name | ( expr )

selection ::= select ( condition ) atomic-expr

condition ::= conjunction { || conjunction }

NOTE: Arbitrarily long conditional expression is optional (3/100 point extra credit). You may implement single comparison, conjunction (and), and disjunction (or) of two comparisons for full credit.

conjunction ::= comparison { && comparison }

comparison ::= operand op operand
                     | ( condition )

op ::= == | != | < | > | <= | >=

operand ::= attribute-name | literal

attribute-name ::= identifier
literal ::= intentionally left unspecified

projection ::= project ( attribute-list ) atomic-expr

attribute-list ::= attribute-name { , attribute-name }

renaming ::= rename ( attribute-list ) atomic-expr

union ::= atomic-expr + atomic-expr

difference ::= atomic-expr - atomic-expr

product ::= atomic-expr * atomic-expr

Queries generated from the above grammar compute new relations based on existing relations. Queries can also name those new relations. We need, however, some ways to create some initial relations (constituting a database), update the relations within the database, store the results of queries back to the database, and delete tuples from relations. We use the following commands for these purposes:

command ::= ( open-cmd | close-cmd | write-cmd | exit-cmd | show-cmd | create-cmd | update-cmd | insert-cmd | delete-cmd ) ;

open-cmd ::== OPEN relation-name
close-cmd ::== CLOSE relation-name
write-cmd ::== WRITE relation-name
exit-cmd ::== EXIT
show-cmd ::== SHOW atomic-expr
create-cmd ::= CREATE TABLE relation-name ( typed-attribute-list ) PRIMARY KEY ( attribute-list )

update-cmd ::= UPDATE relation-name SET attribute-name = literal { , attribute-name = literal } WHERE condition

insert-cmd ::= INSERT INTO relation-name VALUES FROM ( literal { , literal } )
                       | INSERT INTO relation-name VALUES FROM RELATION expr

delete-cmd ::= DELETE FROM relation-name WHERE condition

typed-attribute-list ::= attribute-name type { , attribute-name type }
type ::= VARCHAR ( integer ) | INTEGER
integer ::= digit { digit }

A program in our data manipulation language (DML) is then defined as:

program ::= { ( query | command ) }

Example

CREATE TABLE animals (name VARCHAR(20), kind VARCHAR(8), years INTEGER) PRIMARY KEY (name, kind);

INSERT INTO animals VALUES FROM ("Joe", "cat", 4);
INSERT INTO animals VALUES FROM ("Spot", "dog", 10);
INSERT INTO animals VALUES FROM ("Snoopy", "dog", 3);
INSERT INTO animals VALUES FROM ("Tweety", "bird", 1);
INSERT INTO animals VALUES FROM ("Joe", "bird", 2);

SHOW animals;

dogs <- select (kind == "dog") animals;
old_dogs <- select (age > 10) dogs;

cats_or_dogs <- dogs + (select (kind == "cat") animals);

CREATE TABLE species (kind VARCHAR(10)) PRIMARY KEY (kind);

INSERT INTO species VALUES FROM RELATION project (kind) animals;

a <- rename (aname, akind) (project (name, kind) animals);
common_names <- project (name) (select (aname == name && akind != kind) (a * animals));
answer <- common_names;

SHOW answer;

WRITE animals;
CLOSE animals;

EXIT;

Note that we made a distinction between queries and commands in the grammar of the DML. The result of a query is a view. A view is not stored in the database. Rather, it is a temporary relation whose lifetime ends when a DML program finishes. So only the updates caused by the commands persist from one DML program execution to another.

The relations themselves should be saved in a file in plain ASCII text, using the same DML described above (e.g., CREATE ... INSERT ... INSERT .... ). To make it simple, let us assume that each database file can only store one relation and the filename is the same as the relation name with the suffix ".db". To load a relation from a database file, use the OPEN command. Opening a nonexisting file will result in nothing. To add a new relation to a file, use the WRITE command (the filename will be by default "relationname.db"). To save all changes to the relation in a database file and close, use the CLOSE command.

To exit from the DML interpreter, use the EXIT command.

To print a certain relation or a view, use the SHOW command.

DB Application

The second phase of this project is to write a simple DB application written in the DML described above. The DB app will run on your DB engine. Since you will need to take user I/O and implement a custom control flow (conditional statements, loops, etc.), the DML described above alone is not enough. Instead of extending the DML to include such non-DB commands, you will use a host language (e.g., C++) to interact with your DBMS.

The use of the DBMS from a host program consists of first generating a DML program (as a string), and then sending the generated program to the DBMS to be executed.

You may also need to retrieve the results of the queries to feed into the host language's control flow. The DBMS object can contain a member function to access the relations, views, and the attributes by their name (string). This is what an example interaction might look like:

string name;
cin << name;

string query = string("") + 
               "answer <- project (age) ( select (kind == \"dog\" && name == " + name + ") animals )";

rdbms.execute(query);

if (rdbms.relation(relation_name).int_field(field_name) == 10) {
	...
}

The DB application itself will be a simple DVD rental database:
  1. Customer data: USER-ID, Firstname, Lastname, Phone Number
  2. DVD data: Inventory number, DVD-ID, Title
  3. Rental log: USER-ID, Inventory number, Check out date, Due date
  4. Operations: Add new customer, Add new DVD, Remove customer, Remove DVD, Update customer, Update DVD, List customer, Search customer (by name, phone), List DVD, Search DVD (by ID, title), Search inventory, Check out DVD, Check in DVD, Show rental list by customer, Show customer list by DVD-ID.
  5. The user interface can be a simple scrolling CUI that prints the menu and takes user input from the keyboard.
Note that most of the code in the host language will be user interface, as you will be able to use the DML for most of the required operations.

Deliverables and Requirements

  • Each team should maintain a development log (Google doc) updated by the team members. Give access to the TA and the Instructor. This log will be graded. There is no designated format. We will check your daily progress.
  • Major routines should include unit testing.
  • Demo in the lab may be required.
  1. Design documents: Follow the guidelines in Scott' Hackett's "How to Write an Effective Design Document" (Writing for a Peer Developer). Include all four sections described in the guide.
    • The design document should cover both phase 1 (DB engine) and phase 2 (DB app).
    • Phase 2 documents should include ER diagram and corresponding relation schema, besides other things.
  2. Parser code: Upload your parser code. It should be able to accept or reject an arbitrary command or query: Accept syntactically correct command or query, and reject anything that violates the syntax.
  3. DBMS engine code: Upload your DBMS engine code. It should compile into a stand-alone executable application so that when you run the application, it works as a DBMS command shell, where you can type in any command or query.
  4. Final project code + DB application code + report: The DB application should be a stand-alone executable. The report should be
    • an updated version of your design document
    • post production notes (changes you had to make to your design and why, difficulties, solutions, lessons learned)
    • individual work load distribution (percentage)
    • self-evaluation (briefly discuss how you contributed to the project) and peer evaluation (rate all other team members on the scale of 1 to 5, 5 being best; plus brief comments). Peer evaluation should be submitted individually.
    • Development log.

Submission

All submissions should be through elearning.tamu.edu
Except for the peer evaluation (submit individually), only one person from your team may submit, as a designated submitter. All other submissions will be ignored. See course web page for information on late penalty.

Original concept/design/most of the text by Jaakko Järvi. Modifications by Yoonsuck Choe.