Brief Overview

Okay, so here we go. This is going to be a bit of a rambling adventure, because that’s the only way it can follow what I actually managed to accomplish.

I’ve been obsessed with learning LINQ to SQL since I first heard of it, but never really had the time. Besides time constraints, I was also put off by the fact that there seemed to be nothing coming from the Oracle camp regarding the exciting new technology I saw unfolding. After a while I began hearing whispers of the Entity Framework, a complete object-relational mapping solution coming from the Microsoft camp, and was immediately interested. I had never really taken a look at the other available ORM solutions for .NET, NHibernate and the like, simply because I thought there would be a slim chance I would be able to use both Oracle and LINQ with them without too much trouble. As it stands, I had exactly that trouble with Entity Framework, but it feels justified in a weird way. I’m sure after this I’ll try taking a look around for other available solutions, but so far this looks extremely promising, if a little unpolished.

This all starts off with EFOracleProvider, an Entity Framework-friendly provider used to generate the necessary classes and mapping files for database access. Compiling EFOracleProvider gave me a DLL and some configuration settings I could toss at the already existing data model generator, EdmGen.exe, to generate the classes that map to the database tables and fields I specified.

I’m going to show you the files I worked with in all of this, but I’m not going to obfuscate them, or simplify them. I’m going to give you an idea of the exact situation I was in, in order to show you the problems I encountered, and how easy it is to encounter them. I’d also like to remind you that use of EFOracleProvider is not recommended in production environments, as it is extremely rough quality code. This was more of an adventure than a focused goal, but it raises a lot of questions.

Observing the Database

My first step, after compiling EFOracleProvider, was to create a storage model for the part of the database I wanted to present to .NET. I decided to go straight to our live database and take my queues from it. I took a small cross-section of our available schema — just a single table, with two columns, both VARCHAR2s.

    +--------------------------+
    | tblAsset                 |
    +--------------------------+
    | Asset_Number             |
    | Asset_Type               |
    +--------------------------+

Yes, Asset_Number is a string. It refers to alphanumeric serial numbers. But it’s also our primary key, so it’ll make do. In any event.

The storage model file is XML, and fairly easy to create, but has a lot of hidden gotchas that we’ll cover.

<?xml version="1.0" encoding="utf-8"?>    
<Schema Namespace="TestAppEFModel.Administration" Alias="Self" xmlns="http://schemas.microsoft.com/ado/2006/04/edm/ssdl" Provider="EFOracleProvider" ProviderManifestToken="10g">
  <EntityContainer Name="dbo">
    <EntitySet Name="Assets" EntityType="TestAppEFModel.Administration.Assets" Table="tblAsset" />
  </EntityContainer>	  
  <EntityType Name="Assets">
    <Key>
      <PropertyRef Name="Asset_Number" />
    </Key>
    <Property Name="Asset_Number" Type="varchar2" Nullable="false" StoreGeneratedPattern="Identity" />
    <Property Name="Asset_Type" Type="varchar2" MaxLength="50" />
  </EntityType>
</Schema>

Fairly straightforward. We’re reminded in the EFOracleProvider documentation to set the ProvierManifestToken to the relevant version number of our database. You’ll note that your namespace seems to require a secondary class. You cannot place your entities in TestAppEFModel, you need to create something like TestAppEFModel.Administration. Also note that I’ve manually specified the table name in the EntitySet tag, since it does not map cleanly to what .NET expects.

Fire up EdmGen.exe:

    C:\OracleEFTests>edmgen.exe /mode:FromSSDLGeneration \
        /c:"Data Source=TNSNAME;User ID=user_id;Password=password..." \
        /project:TestApp /prov:EFOracleProvider \
        /inssdl:TestAppEFModel.Oracle.ssdl /project:TestAppEFModel
    Microsoft (R) EdmGen version 3.5.0.0
    Copyright (C) Microsoft Corporation 2007. All rights reserved.

    Creating conceptual layer from storage layer...
    Writing msl file...
    Writing csdl file...
    Writing object layer file...
    Writing views file...

    Generation Complete -- 0 errors, 0 warnings

Preparing the Application

There are two basic steps to preparing your application: Providing a DLL of the object model, and configuring your application for use with EFOracleProvider and your database. Providing the DLL is easy enough — take the two .cs files created by EdmGen.exe and throw them into a project, and reference that project in your app.

I had initially put all of the test and generation files in one root directory on my disk, just to keep track of them. This turned out to be a good idea for a secondary reason. In your application configuration, you must provide access to the metadata files you’ve just created. The example code in the EFOracleProvider project used the following syntax in its App.config:

<add name="NorthwindEntities"
        connectionString="provider=EFOracleProvider;metadata=res://NorthwindEFModel/NorthwindEFModel.NorthwindEFModel.csdl|res://NorthwindEFModel/NorthwindEFModel.NorthwindEFModel.msl|res://NorthwindEFModel/NorthwindEFModel.NorthwindEFModel.ssdl;Provider Connection String='data source=XE;user id=edmuser;password=123456'" providerName="System.Data.EntityClient" />

Note those res:// addresses. I imagine those are used to refer to resource files, but I didn’t see any of the projects involved actually containing one. On top of that, there’s that double-namespace thing, i.e. NorthwindEFModel.NorthwindEFModel.csdl. I couldn’t get this syntax to even work a little bit with my configuration, so I just pointed my application at the root directory I created, that contained all of the necessary files. I imagine on a test or application server, leaving these files in a common place may be the most useful option.

<add name="TestServer"
    connectionString="provider=EFOracleProvider;metadata=C:\OracleEFTests\TestAppEFModel.csdl|C:\OracleEFTests\TestAppEFModel.msl|C:\OracleEFTests\TestAppEFModel.Oracle.ssdl;Provider Connection String='Data Source=TNSNAME;User ID=user_id;Password=password...'"  />

Don’t forget to add EFOracleProvider to the <DbProviderFactories> section of your configuration, as well. Also note that I didn’t use the providerName attribute, which in the test code was listed as System.Data.EntityClient, even though the provider in the connection string is our good old EFOracleProvider.

Finally, write a small piece of code to test the functionality.

using System;
using System.Collections.Generic;
using System.Data.EntityClient;
using System.Linq;
using System.Configuration;
using System.Text;
using TestAppEFModel;
 
namespace TestAppEFOracleTests
{
    class Program
    {
        public static void Main()
        {
            TestAppEFModelContext context = new TestAppEFModelContext( ConfigurationManager.ConnectionStrings["TestServer"].ConnectionString );
 
            var query = from asset in context.Assets
                        where asset.Asset_Type == "TEST ASSET TYPE"
                        select asset;
 
            foreach(var asset in query)
                Console.WriteLine("{0}, {1}", 
                    asset.Asset_Number, 
                    asset.Asset_Type);
        }
    }
}

The Troubles

And we’re good to go, right? Not quite.

ORA-00942: table or view does not exist

Whoops. What did we do wrong? Well, let’s look at the SQL that was generated.

SELECT
    "Extent1"."Asset_Number",
    "Extent1"."Asset_Type"
FROM
    "tblAsset" "Extent1"
WHERE
    (N'TEST ASSET TYPE' = "Extent1"."Asset_Type")

What’s wrong with that? Actually, something very subtle and dangerous that I’m sure is causing Oracle a lot of pain in trying to get a provider out for the Entity Framework. Observer the following session in SQL*Plus:

SQL> DESC tblAsset;

Name           Null?    Type          
-------------- -------- --------------

ASSET_NUMBER   NOT NULL VARCHAR2(50)  
ASSET_TYPE              VARCHAR2(50)  

SQL> DESC "tblAsset";
ERROR:
ORA-04043: object "tblAsset" does not exist

SQL> DESC "TBLASSET";
Name           Null?    Type          
-------------- -------- --------------

ASSET_NUMBER   NOT NULL VARCHAR2(50)  
ASSET_TYPE              VARCHAR2(50)  

This hearkens back to a long-standing conflict in dealing with Oracle and mixed-case or reserved identifier names, to wit: In order to ensure that table names that happen to be reserved keywords can be used, we need to enclose them in double-quotes. But we can create tables with or without quotes, with differing cases, and create two different tables. We may not be able to rely on the user telling us the exactly correct name of a table in our storage model file, but we can’t convert it to all caps without the risk of accidentally referring to a different table!

My solution at this point was to cave and refer to the table name in all caps in my storage model file. But now the problem is the column names! And we can’t refer to a column name in the storage model, but we can in the mapping file, the .msl, but we can’t simply generate the mapping file, change the column names to all caps, and have it compile. What I ended up doing was simply referring to all of the fields in all caps in the storage model, which of course breaks our code, making us refer to asset.ASSET_TYPE, as well as uglifying the constructor code for creating new objects:

/// <summary>
/// Create a new Assets object.
/// </summary>
/// <param name="aSSET_NUMBER">Initial value of ASSET_NUMBER.</param>
public static Assets CreateAssets(string aSSET_NUMBER)

One More Thing

So, those are the big things, the deal-breakers. The largest problem to me is the idea that we can specify a table name in our storage model file, but not column names. This seems entirely too half-baked to me, but maybe I just don’t understand the conceptual tactics behind a move like that.

And I just noticed this as I was signing off, a quick bizarre thing I wanted to try:

var query = context.Assets.Where( a => a.ASSET_TYPE.Contains("A") ).Take(10);

Take the top ten items that have an uppercase A in them. Simple enough, right? Nope. A rewriting of the generated SQL:

SELECT 
    asset_number, 
    asset_type 
FROM 
    tblAsset 
WHERE ((INSTR('A', asset_type)) >0) 
AND   ( ROWNUM <= (10) )

This is a mess. First off, that ROWNUM clause is not going to do what we expect, no matter what the rest of the query is. Secondly, INSTR takes its parameters in the reverse order. Little things like this are going to be the hell Oracle would have to go through to get a decent provider out there for Entity Framework, and I don’t even envy them a little bit.

One more thing I want to say about all of this. The work I did above took seven hours, a lot of Googling, a lot of searching through documentation, and a lot of guessing. The following is how everything above would be handled in my current favorite Ruby ORM.

class Asset
  include DataMapper::Resource
  property :asset_type, String
  property :asset_number, String
end
 
query = Asset.get :asset_type => 'TEST ASSET TYPE'

This is not to compare languages or ideologies, but methodologies. Simplicity always trumps complexity in the software engineering world. Always always always. Make Entity Framework easier for people to use, and it could be a godsend for shops that want a Microsoft-written ORM solution.

I’ll try and answer any questions anyone has, but this is literally the maximum scope of what I was able to accomplish. I’ll probably throw myself back at it later this weekend.

This is the first release of an evolving library that interacts with the online todo-list thingie Tasko. If you’ve never heard of it, or its brother-in-spirit TaskPaper, you’re in for a treat.

Plain text files, converted to To Do lists with simple formatting rules. Lines that end with a colon are project names. Lines with a hyphen starting them are tasks. Use tags on lines to categorize them. @today, @home, whatever. Tag a line @done to cross it out. Simple!

So now we have Taskomaly, which works on your local machine, interacts with task lists, and can upload them to Tasko using its API. Tasko provides both an API key and a user ID when you register. You need both of these things. Register, then go to your settings page to retrieve them. I suggest placing them in a file in YAML format, like so:

  user: 9999
  key: d9cca721a735dac4efe709e0f3518373

Then, off to irb!

  t = Taskomaly::From '~/.tasks.yml'

Now, get the information you want. Your papers! Make a change, then save them back to the website.

  t.request :papers # ['Paper One', 'Paper Two']
  p = t.papers.first
 
  puts p.name # 'Paper One'
  puts p.body # duh
 
  p.body["@today"] = "@tomorrow"
  p.save # true if groovy, false if failed

This is the first real project I’ve put a huge emphasis on TDD, guided along by the simple principles espoused by Adam Wiggins during his work with the rush shell.

So, yeah. That’s that. Work (and specs) are coming along soon to help you with individual projects inside task papers, searching and using tags, and all kinds of other fun stuff.

EDIT Jesus I forgot to put a link to the project. Here it is. On GitHub. Duh.

Stagger in disbelief if you’d like, but I’ve found an instance where it is more beneficial to not begin testing a piece of software until a measurable way through its development.

You no doubt remember my fabulous, in-development command-line NewsGator client, bulletin. I’ve extracted the NewsGator-specific code to a gem called WonderCroc. The README provides a small glimpse of what’s possible with the library now:

wondercroc

A gem that provides…

In any event, I haven’t written any tests for it yet. A lot of the initial programming was screwing around with Ruby’s RSS parser and running requests through in IRB. At this point, however, I’d like to start throwing specs at it as I refactor it and extend it to become a modular library.

So what?

Well, here’s the thing. I want to know that the library responds correctly to the XML responses sent by NewsGator’s server. I assume the correct way to test this is to mock the responses accurately. The NewsGator REST API documentation provides several partial samples, but they’re not complete or complex enough to give me a full range of parsing opportunities. So why not use actual responses?

So that’s my plan. Extend my WonderCroc::Client class to log the XML responses to a file, and use those in my mocks — something I can’t think of a good way to do if I hadn’t written the code in the first place.

Thoughts?

If you’re doing what I’m doing, and need to parse an RSS feed that has lots of fun little tags in other namespaces you want to slurp up along with all the normal things, here’s something you can do.

We’re going to use the example I’ve been working on, a) because it allows me to point out an interesting problem, b) because it allows me to brag about what I’m working on, and c) because at this point I’m too tired to think through the logic of making an example work.

I’m writing a command line based feedreader called bulletin in Ruby. bulletin uses NewsGator to sync online feeds. Here’s an enticing, exciting pre-release preview screenshot!

In any event, there’s lots of cool metadata in NewsGator’s RSS feeds. The one piece I was interested in was whether or not an item in a feed has been read by the user. It appears in the feed as this element:

<ng:read>True</ng:read>

Awesome. So how do we go about getting this item and parsing it like it ain’t no thang? By extending Ruby’s RSS parser, like so.

First, we extend the Item class for RSS feed items to add an extra attribute:

module RSS; class Rss; class Channel; class Item
  install_text_element "ng:read", "http://newsgator.com/schema/extensions", '?', "read", :boolean, "ng:read"
end; end; end; end

Here’s what this means: We want a new element, that looks like ng:read. It comes from this schema: http://newsgator.com/schema/extensions. We don’t know where it will show up in the parsing of an item (?). The name of the attribute we will access it with is read. It’s a :boolean type. If we write an RSS feed back out, it will appear as ng:read in that feed.

That is, I think that’s all true. This is a lot of experimenting and diving through source.

Next, we tell the parser to look for another element:

RSS::BaseListener.install_get_text_element "http://newsgator.com/schema/extensions", "read", "read="

This says: Install this element into the parser. It comes from this schema: http://newsgator.com/schema/extensions. Its accessor method is read. It’s setter method is read=.

And then you’re good! Well, except for one thing.

The name of this particular element, less its namespace, is read. The Listener needs to know what to call its accessor and setter methods. That means some reflection magic is being done behind the curtains. Yes! So now you have to be extra careful with this particular Item, because now its original read method has been overwritten. All three times we have a parameter up there with read have to be the same. I haven’t gotten it to work any other way.

The implications:

• I haven’t found a way to give an element accessors and getters that are not its element name without the namespace.
• Printing the item back out with to_s doesn’t appear to bring the new element with it, although from the looks of it my method above doesn’t provide for that no matter what the element is named.

I’d love to talk to someone who knows the internals a bit more — or at least someone who could help me write some documentation for Ruby’s RSS parser. This is a pretty important thing and it would be awesomely useful.

In the meantime, have fun with your newfound knowledge! We now have an Item#read method that gives us true or false, depending on what was parsed.

Let me know if you make any progress in figuring this beast out.

At one point I had signed up for the Things alpha, a very nice looking To Do list app. I made some notes regarding it, and I thought I’d throw them up here for the general benefit. As it stands I’m quite happy with my plain text To Do list solutions. Portable, flexible, and cross-compatible.

So! My notes:

• Star icon looks out of place, looks anti-aliased when all the other icons have very fine details.
• Eyes try to gloss over the inset introductory text, try making all three points on one line each, widening the box if necessary…
• If Things is supposed to make things easier for users, don’t give them a huge white chunk of data entry when they create a new To Do. Tags and notes could perhaps be simple icons that expand when clicked on, or if the key is pressed.
• No import/export, though it is noted in the alpha notes that this is coming
• No right-click menu on the trash folder
• Would like to be able to minimize ‘collect’, ‘focus’, and ‘organize’ sections
• ‘File’ dialog box feels cramped.
• If no Projects or Areas are created, the only option in the ‘File’ dialog box is to cancel. Perhaps a warning if a task cannot be filed at all, or the file icon on the toolbar could not appear?
• Dragging a task to the root Projects section on the sidebar to promote the task to a project seems counterintuitive - I see how this could be beneficial, but perhaps some obvious mention of that ability would be good a Promote to Project button? I would hate to sully the interface with more buttons…
• What’s the difference between a Today task and a task in the inbox or some other area that is due today?
• I may be very oblivious, but I can’t seem to find a way to delete a project. Not implemented yet? (Finally found it. There needs to be a Command-Delete shortcut with confirmation.)
• My first instinct on wishing to create a new task was to double click in the main grey area of the window. I’m not sure how many other people have this impulse, but using that as a target to create a task instead of trying to aim for the (relatively) small ‘New task’ button on the toolbar might be beneficial…