DB Schema: Project Outline

Table of contents for DB Schema

  1. DB Schema: Project Outline

So here it is the first ever series. It’s been a while since I posted the introduction but the posts should be coming thick and fast now I’ve chosen my first subject, DB Schema. This is something we’ve recently implemented at work and, if I do say myself, it has been reasonably successful. In this first post I’m going to outline what I want the project to achieve.

I think it would be amiss not to have a quick scan of what’s already available. There are various implementations of this sort of functionality like Ruby on Rails migration, Phing DBDeploy Task and eZ Components DB Schema. Looking over the eZ components DB Schema documentation page shows they’ve done an extensive amount of work. Their implementation should fulfil any developers possible need. I only intend to implement a subset of what they have.

Zend Framework currently has a couple of proposals drafted but as yet no actual activity on this front. I’m hoping that the final product of this series will be acceptable as a contribution. This means that everything I go through here will be compliant with ZF’s guidelines.

My goals for this series are to provide a working implementation, the unit tests and documentation for the final submission of the proposal to the Zend Framework. As such I will be using components from the Zend Framework, specifically the DB classes and the functionality they offer. My objectives for the implementation is to provide a set of loosely coupled classes which a developer can take and mould to their specific needs.

The classes will provide a vendor neutral fluent interface to allow the developer to quickly create database schema changes which can be applied across multiple databases and servers. The problem with abstraction layers is that they generalise, so the interface needs to allow the developer to access the specific functionality easily when it’s needed. There are two further areas of requirements for this project. The first is the schema patches need to be able to downgrade the schema as well as upgrade. If there’s a major problem with your application and you have to downgrade the code, you’ll want to downgrade the database as well.

Lastly is the need to be able to handle errors gracefully. At first this seems obvious, shouldn’t all code handle errors gracefully? What I mean by this is that you don’t want to update your production server and find the database update dies half way through it’s changes. A nightmare ensues trying to figure out why the update failed, how far it got and how to fix the tables to make sure your application code keeps running. This means that the classes have to try to handle errors and undo changes where possible. There also needs to be a detailed log of the actions that were taken an succeeded, the action that failed with the reason and the steps that were taken to rectify the problem, if any. I’m not sure how successfully this can be implemented but it’s something I want to strive for if possible.

Proposal requirements:

  • code
  • unit tests
  • documentation

A quick round up of project requirements:

  • flexible architecture
  • fluent interface (with code completion/hinting for IDEs)
  • access to the database adapter
  • flexible patch ordering system
  • flexible patch syntax (PHP and SQL)
  • ability to upgrade and downgrade
  • error handling and reporting

That concludes the first part of this series. Hopefully I’ve outlined everything I want the final code to do and my expectations. In the next part I’ll be looking at breaking the requirements down, setting up the project structure and creating a few of the initial unit tests.

A (not so) quick Introduction

Hi and welcome to my blog. This is the first post of many to come.

In this initial post I’d like to cover some background about the projects I’m working on and what components I am likely to be covering in forthcoming posts.

A small note about the structure of the blog. I plan to analyse specific and unique components from the different projects I am working on. I want to demonstrate, learn and refine the entire process. The one thing this does in presenting the component in the manor is to make it a very specific detached unit. This is because I will be presenting the idea and development outside the context of the project itself. This means I’m already meeting one design goal OO by separating concerns. I won’t go further into this but suffice to say that when applying the process in your own projects it may be helpful to design and implement code in this frame of mind. In fact this is one of the founding principles of TDD.

Onto the projects. I have four projects I am currently involved on. The main one being the one I work on full time as part of my job. It’s fantastic project to be working because it presents so many challenges and interesting avenues of development. The other projects include a small web site for a sailing club and two open source projects of which I’ll go into more detail on below.

Maxemail (Work)
Maxemail is an email marketing system. In it’s simplest form it allows you to upload a list of email addresses and send a message to them. Then information is fed back to the system about who opened the email and clicked on a link and that information is compiled and presented to the user. There is a large amount of extra functionality formed around this core system. Some of this functionality or rather the components will be a focus for discussion here.

The latest version of the software has recently been released. Version 5 of the Maxemail product is based on the Ext JS javascript interface library which interfaces with a PHP backend through JSON calls. The setup uses a replicated MySQL setup with round robin load balanced application servers. This architecture makes the Maxemail system itself highly scalable.

Sailing Website
Overall web sites don’t really excite. They tend to be basic static HTML files with some dynamic content with PHP thrown in. The sailing web site I am in the process of redeveloping is no different. However there are few areas of interest which are a little beyond the norm of run of the mill web site.

Open source projects
These are two projects that have been started only recently. One is WebExplorer and the other is ExtDb. At the moment the ExtDb is on hold while work is underway on the WebExplorer project.

WebExplorer
The web explorer project aims to provide an interface using Ext JS to navigate most file systems. For example to most obvious ones that spring to mind are local and ftp but thinking further we’d like to integrate other file system like services. These might include SCP, Amazon S3 and maybe even network based shares. These would all be provided in one windows explorer like interface. The are no firm plans yet but there is potential for the project to incorporate other file operations other than just navigation.

ExtDb
ExtDb is a large and very ambitious project. The aim of the project is to provide an interface similar to SQLyog, again using Ext JS, for multiple databases. The main features would be CRUD for tables, indexes and views. The goal is to replace phpMyAdmin as the default administration interface. I think you’ll agree, a lofty goal indeed.

As I work on these projects I’ll present different aspects of the design and development here.