I recently decided to take a break from my own, somewhat-custom, python, web framework to play around with Django. I’ve been doing some contracting work, and the most popular web framework for Python seems to be Django. I decided that it would be worth my time to actually build something real using it (other than just a tutorial) to get some deeper familiarity with it (and maybe to lift ideas for PyBald).

There are a lot of things to like about Django, I’m saving those up for another post. This post is about how I just burned several hours trying to do a few things that seemed like they should be simple but turned out to be oddly frustrating at times. Im sure that part of this is just not being Django-expert-enough to have seen the obvious answers but the task at hand seemed simple enough: creating a user registration form for my application. (Yes I know there’s a user registration django app, but this seemed like a trivial task that should be easy with a good framework)

Custom Forms and Error Display

I started down the rabbit hole when I decided I didn’t really like the default way Django renders forms. There are a couple of options (as_table, as_p, as_ul), but whether displaying as a table, a list, or as <p> tags, the form layout didn’t match what I was used to. I know tables are often used for layout, but I really try and keep them for tabular data whenever possible. I also generally like to reserve <p> tags for actual text/content paragraphs and prefer <div> tags for logical divisions in a document. Might be nit-picky, but that seems more semantic to me.

I could have lived with one of the default renderings but I found that I also wasn’t crazy about the way Django handles validation error display by default. Normally Django spits out a <ul> list of field errors with a css class of “errorlist” right above the offending field (or at the top of the form for general errors). Since the errors are only tied to the field by proximity, there’s no way to call out the offending field directly. With some styling, I could have made the error list clearer, but I still wasn’t crazy about this pattern.

When designing forms, I like to provide a small amount of usability by highlighting the error field in some way with an error style. At first I could find no way to do this with the default form rendering. Later I figured out that you can modify the widget tied to the field, but that seemed like putting too much display logic in the controller. It also seemed to require a lot of code repetition, putting this piece of code anywhere a form was instantiated and validated. I thought about creating a custom subclass of the form but that seemed heavy handed and I definitely didn’t want to alter Django internals. It also required playing with the two different kinds of field objects that come out of a form (more on that later).

for field in form:
     if field.errors:
        f.fields[field.name].widget.attrs['class'] = "field_error"

I wanted to keep this logic in the template which meant I needed to define my own form rendering. Easy enough, I just wrote my own generic form using the Django template language. I generally put input fields inside <div> tags. I also add a field_error css class to the divs that contain inputs with errors. That lets me define both special styles to highlight that section of the form and the offending fields directly ( with css like: .field_error input {} ), as well as use the error class as a key for special effects (like jQuery fades, etc…).

My initial custom form looked like this:

{# Include the hidden fields in the form #}
{% if form.non_field_errors %}
<div class="form_errors">
  {% for err in form.non_field_errors %}
  <div class="form_error_message">{{ err }}</div>
  {% endfor %}
</div>	
{% endif %}
{% for hidden in form.hidden_fields %}
   {{ hidden }}
{% endfor %}
{% for field in form.visible_fields %}
  <div {% if field.errors %}class="field_error"{% endif %}>
    {{ field.label_tag }}
    {{ field }}
    {% for err in field.errors %}
    <span class="error_message">{{ err }}</span>
    {% endfor %}
  </div>
{% endfor %}

Now from the Django documentation, they suggest creating a custom inclusion tag to render forms like this, but I held off on that for now (although it looks fairly easy). Since I’m generally producing one form per page (at the moment) I’m just using the Django include tag and including this form template into my page template where I need a form. So instead of {{ form.as_ul }} I’m using this: {% include base_form.html %} and assuming I’m using a context variable of form in the outer template (or using ‘with‘ to override form in the include as described in the docs).

Getting my own error displays working, with field highlighting I was happy with, was fairly easy. Right now validation errors appear as a series of spans next to the input field, but I’ll probably change this later.

Error css applied in the display logic. A simple yellow bg color styles allow me to highlight the error and the field.

Custom Form Validation

Next I started working with validation. Since this is for a custom user registration form, the first thing I need to do is validate that the username being registered is unique. Django has the concept of validation functions that can be passed to form fields. For my own web framework I’ve been using FormAlchemy, which has a fairly robust validation system and Django’s system is similar in some respects.

def validate_username_unique(value):
    '''Custom validator for user uniqueness.'''
    if User.objects.filter(username=value).exists():
        raise ValidationError(u'Sorry, someone already has that [...]')
 
class UserRegisterForm(forms.Form):
    # [... stuff ...]
    username = forms.CharField(validators=[validate_username_unique])
    # [... stuff ...]

Nice and clean, this is how I expect form validation to work.

Next I needed to confirm that the password and password confirm fields match. Validators only take one value input, so how do I validate that the fields match? Turns out Django’s validators aren’t up to the task so you have to plumb a little deeper into how it validates forms (as a note, Django’s docs are very good, but often times the difficulty is figuring out where in the docs something may or may not live).

To validate based on two fields you have to override the clean method on your subclassed form. When you want to flag validation errors on a particular field, you set the _errors['FIELDNAME'] on the form to flag the invalid fields. (remember that the clean method must return the cleaned_data member. That bit me in the rear for a while.)

class UserRegisterForm(forms.Form):
    # [...stuff...]
    password = forms.CharField(widget=forms.PasswordInput)
    password_confirm = forms.CharField(widget=forms.PasswordInput)
 
    def clean(self):
        '''Required custom validation for the form.'''
        super(forms.Form,self).clean()
        if 'password' in self.cleaned_data and 'password_confirm' in self.cleaned_data:
            if self.cleaned_data['password'] != self.cleaned_data['password_confirm']:
                self._errors['password'] = [u'Passwords must match.']
                self._errors['password_confirm'] = [u'Passwords must match.']
        return self.cleaned_data

So now I have a form that checks that two fields are equal or else it flags a validation error. I had to add validation logic using two different mechanisms which seems a little messy but not too bad.

Passwords must match!

Fieldsets

Lastly I decided I wanted to split my form inputs into fieldsets. Generally this is considered good practice, especially for usability and accesibility. The first option would be to just write out the form by hand, including the fieldsets, but that seemed like it could be brittle and not very django-like. Django also has some nice fieldset functionality for it’s admin app, so I assumed this would be trivially easy. Google searches seemed to turn up overly complex solutions for something that seems like it should be ‘built in’. There’s also a library ‘django form-utils’ that has fieldsets but I was trying to stick with generic Django forms.

This is where my frustration level really started to rise. I became very annoyed at the philosophical position that Django has taken that Django templates can’t run arbitrary python code. There are arguments for why this is/isn’t a good idea in a template system, but having come from using Mako for my templating system, this limitation started to drive me a little crazy.

My first impulse was to create a one-off custom form and use display logic in the template to change how things were laid out.

<fieldset><legend>Name</legend>
{% for fieldname in ('prefix','first_name','last_name') %}
{# render the fields for this fieldset #}
{% endfor %}
</fieldset>

Bzzzt.

TemplateSyntaxError at /register/user
Could not parse the remainder: '('prefix','first_name','last_name')' from
'('prefix','first_name','last_name')'

Django doesn’t allow you to create tuples or lists inside for blocks. The for tag only seems to work on iterators passed into the context. This seemed a little annoying to me since this logic seems ideally suited as display logic but it was forcing me to move the logic into the controller logic. I monkeyed with all sorts of ways to try and modify the display logic to no avail.

Then I decided to create an iterator that I could pass into the context so I could call out the individual field names.

    fieldset = ({'label':'Name','fields':('prefix','first_name','last_name')},)
    return render_to_response('registration/user_register.html',
                             {'form': f,'fieldset':fieldset}, 
                             context_instance=RequestContext(request))

Then in the template I tried using the names on the fields dictionary.

{% for set in fieldset %}
<fieldset><legend>{{ set.legend }}</legend>
{% for fieldname in set.fields %}
{{ form.fields[fieldname] }}
{% endfor %}
{% endfor %}
</fieldset>

Nope: TemplateSyntaxError. Again, the Django template language doesn’t like you accessing dictionary values by name in a variable block. Attributes seem OK, key values no. I’m not sure I like this “echoes of python” approach in the template language because it means learning another logic system rather than applying the full expressiveness of Python.

So, no dictionaries, I’ll pass the fields themselves in the fieldset as iterable objects.

fieldset = ({'legend':'Name',
             'fields':(f.fields['prefix'],
                       f.fields['first_name'],
                       f.fields['last_name'])},)

{% for set in fieldset %}
<fieldset><legend>{{ set.legend }}</legend>
{% for field in set.fields %}
<div>
{{ field }}
</div>
{% endfor %}
{% endfor %}
</fieldset>

Alright, no TemplateSyntaxErrors, but wait, what the…

It took a little experimentation and some object introspection but the issue here is that the form actually contains two representations of fields, bound and unbound. The code: for field in form.fields returns different objects than: for field in form.

Poking around the Django core I found that the iterator for a form instantiates BoundField objects from it’s internal fields and returns those. That’s what gets rendered as HTML. The docs do talk a bit about this distinction, but it’s mostly in passing and mentioning you have some additional methods on BoundFields.

Ok, so knowing I need to pass in an iterator, and that the iterator must return BoundFields to properly render in the template, I came up with this FieldSet class.

from django.forms.forms import BoundField
class FieldSet(object):
    def __init__(self,form,fields,legend='',cls=None):
        self.form = form
        self.legend = legend
        self.fields = fields
        self.cls = cls
 
    def __iter__(self):
        for name in self.fields:
            field = self.form.fields[name]
            yield BoundField(self.form, field, name)

So now in my ‘view’ code I instantiate FieldSets and pass them into a new form tempalte that knows what to do with them.

    fieldsets = (FieldSet(f, ('prefix','first_name','last_name'),
                        legend='Name',
                        cls="form_name_info"),
                FieldSet(f, ('username','email'), 
                        legend="User Info"),
                FieldSet(f, ('password','password_confirm'), 
                        legend="Password") )
 
    return render_to_response('registration/user_register.html',
                          {'form': f,'fieldsets':fieldsets},
                          context_instance=RequestContext(request))

Then I wrote an alternate form template with fieldsets to include when I want to use these fieldsets:

{# Include the hidden fields in the form #}
{% if form.non_field_errors %}
<div class="form_errors">
  {% for err in form.non_field_errors %}
  <div class="form_error_message">{{ err }}</div>
  {% endfor %}
</div>
{% endif %}
{% for hidden in form.hidden_fields %}
   {{ hidden }}
{% endfor %}
{% for set in fieldsets %}
<fieldset {% if set.cls %}class="{{ set.cls }}">{% endif %}
  <legend>{{ set.legend }}</legend>
  {% for field in set %}
    <div{% if field.errors %} class="error"{% endif %}>
      {{ field.label_tag }}
      {{ field }}
      {% for err in field.errors %}
      <span class="error_message">{{ err }}</span>
      {% endfor %}
    </div>
  {% endfor %}
{% endfor %}
</fieldset>

And finally: voila, a form with fieldsets generated on the fly, with custom rendering, custom validators, and styled error fields.

Behold! a, well, rather unremarkable form.

Is this the best way to do it? I don’t know, but by the end I was just glad I got at least something to work. This was a first pass so I’m sure I’ll modify this over time (for example, it would probably make sense to create some kind of a FieldSet collection object that could render out a default fieldset for fields not tied to a FieldSet object). I found this day of frustrations definitely dampened my Django enthusiasm a bit. I’m sure as I understand and accept more of Django’s design (like no python code in templates) it will get easier to work with. My initial impressions of Django still stands, it makes getting up and running very fast and easy, but it can get thorny when you deviate from ‘the path’ at all.

I’ve written before about Google’s strategy regarding Android but there seems to be a side effect to this strategy I hadn’t anticipated: the blistering speed with which they are innovating their core platform. The number of Android releases in such a short period of time, and the quality improvements in each release, has been nothing short of amazing. I had assumed iPhone experience parity would take much more time to achieve.

Google’s free (both as in speech and as in beer) approach has allowed them the latitude to focus completely on the core of their platform, to the exclusion of almost everything else. Since they have an odd upper-hand (handset makers really have no other choice) Google has almost completely ignored their needs and requirements. This has left handset makers (and customers) somewhat in the lurch, trying to keep up with the pace.

Android handset customers seem resigned to the fact that when they buy the latest and greatest phone, two months later something newer, shinier, and faster will be available. An interesting part of this approach is that since support is left almost completely to the phone makers, failures in the handset experience (platform fragmentation, slow speed of Android updates, etc…) seem to be blamed primarily on the hardware makers and carriers and not Google. Not being directly coupled to the market gives Google tremendous freedom to continue on their development trajectory.

I think this intentional rocket-ship paced speed of development is not sustainable. I predict that the handset makers (and customers) will begin to complain, and that the dreaded Android fragmentation problems will finally start to appear. The main problem won’t be between different capabilities among handsets (although that will be a problem), but by the fact that there will be a dozen different Android versions with different capabilities all in the market simultaneously.

I think Google’s strategy is to bring the Android experience close to the iPhone’s (it some ways it’s already there or superior) and then have the pace of development plateau. Not because Google won’t be capable of innovating at the same rate, but because market realities will inevitably begin to intrude into the process.

I think several things will conspire to slow down the rate of new versions. I predict Google will have to shift some resources to testing and compliance, creating some form of Google ‘certified Android compatible’ testing lab not unlike what Microsoft had to do with Windows. Handset makers will begin to resist the current pace of innovation by introducing new phones with versions of Android other than the latest. Carriers will continue to drag their feet updating their phones to the absolute latest Android. Unless the way handset makers and carriers do business fundamentally changes I think they will be unwilling to bear the cost of supporting and testing so many different versions of the platform. They will demand some ‘breathing room’ to allow their investment in the current version to pay off before moving to the next.

Another possibility is that Google will continue to innovate along the same curve and intentionally not care about the business realities of their hardware and network partners. Since Google primarily wants the ad revenue from Android on mobiles, maybe they will take a hands off approach and let the handset makers figure out how to make their business models conform to the new realities of Android. The only problem with that is the overall mobile phone experience could be compromised. If there’s one thing the iPhone still has going for it is it’s integrated experience.

Yes, you. Now don’t get defensive, there’s a reason why I know you are terrible at it, you are a human being. So pardon me when, as an entrepreneur, I dismiss your description of my startup as a “dipshit company”, especially when it’s a judgement based on a five second cursory examination. Everyone likes to think they know a good idea / good startup when they see one. Everyone is wrong.

Here’s another secret: the people who seemingly make their living by evaluating startup ideas, they’re terrible at it too. You know how I know? They’re human beings too.

“Wait a second”, I hear you counter, “what about all the money made by venture capitalists?” Well there’s a phenomenon that deserves a little attention, but the key is, they don’t actually make their money by evaluating startup ideas. VC’s have learned to make money from startups with a system. A system not that unlike a gambling system. VC’s use a large bankroll (like professional gamblers) and evaluate factors that tip odds of success (and especially really big success) slightly in their favor. The idea is not one of them. I think in their heart of hearts, most VC’s know they’re no good at evaluating startup ideas. In fact, I would argue the most dangerous and inept VC’s are the one’s that think they can evaluate what a good idea is (and I think this is why Angel investors traditionally lose money).

Here’s my last secret: those crazy sons’o'bitches that are pouring their heart and soul into starting something? They’re no good at judging startup ideas either.

Hey we’re human too. What we do have is passion. We’ve also learned that by doing things, by learning through experimentation, we can figure out why our idea sucks (because 99% of the time it does) and how to make it better. This is the core of the lean startup movement, and historically how the majority of companies have risen to greatness (with something *other* than their initial idea).

This mini rant was inspired by Fred Wilson; “Lead Investors, Dipshit Companies, and Funding Every Entrepreneur“. When we were building Grazr, all those years ago, it was intensely frustrating all of the drive-by-idea-critiques that never bothered to dig into what we were really offering. We made a ton of mistakes, but the initial idea wasn’t one of them.

Here’s the part I really like that resonates with me:

Mike Arrington expressed the contrary opinion, apparently held by many VCs (not me), that this mini explosion in angel investing is creating a bunch of “dipshit companies.” I don’t know what a dipshit company is. I haven’t seen one. If you listen to the chatter on the Techcrunch comment threads, you will see that people think Twitter and Foursquare are dipshit companies. Fine. Many great companies have been built on a wall of derision and I personally think those two are going to join that list of laughed at great companies (and maybe already have). My point is you just don’t know what is a crazy idea and what is a brilliant idea. And you don’t know what is a great team and what is a weak team. Of course, we have our opinions on that. We make those judgment calls every day. But we are often wrong. VCs are wrong more often than they are right. It is good for VCs if 10x or 100x companies get angel funding. That is more opportunity for us.

Always love Fred’s perspective.

I stumbled across a post by Steve Blank of the Lean Startup movement and “Four Steps to the Epiphany” fame talking about business plan competitions. Considering I’ve had my own misgivings about them, I thought it was interesting to read another take on what may be wrong with these kinds of competitions. Steve’s argument is that business plan competitions are mapping the success methodology of big companies onto startups: operation and execution on known opportunities.

Almost every university, region and car wash now has a business plan competition; the rules, who can participate, how large the prizes and who are the judges vary by school.

Business plan competitions perpetuate everything that is wrong about trying to make plans that were designed to be used in large companies fit startups. (One of my favorites: “Judging will include such factors as: Market opportunity, reward to risk, strategy, implementation plan, financing plan, etc.”) All of which may be true in large companies. But little of it is relevant to the chaos and uncertainty in the life of a startup.

– No One Wins In Business Plan Competitions « Steve Blank

He proposes an alternative kind of competition: The Business Model Competition. This kind of competition would focus on startups actually building their product or testing their business ideas in some way to learn what their real business model should be. To me, that seems to map much more closely to the actual process of building a startup.

For the first time we’d have a competition that closely resembled the reality that founders face, rather than a creative writing exercise.

Note: I have not been a part of day-to-day operations at Grazr.corp for over a year now and am not speaking for the company in any way.

Even though I’m no longer with the company, I’ve been asked by several people now what’s happening with Grazr. Sadly, Grazr.com is currently in the process of shutting down. Grazr the company has no official blog anymore so I’ve decided to write about the news. The truth is Grazr and the Grazr Widget have not been a focus of the company for a long time and it appears that the company can no longer justify the expense of keeping it running. Even with the low costs of servers and hosting these days, it still costs something to keep the service up, fast, and reliable.

All Grazr widgets across the web are now displaying a message that the service will cease to function on July 31st.

Now, as the original author of the Grazr widget, to say this makes me sad is an understatement. I’ve toyed with the idea of writing a new service + widget, one that performs many of the same functions of the original Grazr widget, but I decided for various reasons it wasn’t right for me to do it. Now that the company has officially decided to shut down Grazr and the widget, it’s gotten me thinking about putting something together again, primarily as a side hobby.

I’m writing this post partially to gauge interest in such a project. Even as a side hobby, I’d need to explore how this new RSS/OPML service could be made sustainable, some kind of a pay component. I’m not sure exactly what that would be, but at a minimum it would need to cover the cost of the servers. Realistically I can’t afford to run this new project out of pocket so I would need some kind of support to make it happen. I’m interested in ideas as well as what premium services people would be interested in.

I know a common response will likely be “advertising”, but advertising is a trickier business model than most people realize. Even if I were to pursue an advertising model, starting this new service from scratch I can’t afford to pay the initial ramp up period before that kind of business model could potentially become sustainable.

More than likely, this will mark the end of the “super” RSS/OPML widget. I don’t think anyone has really tapped into the deep potential in the combination of OPML and RSS. The nature of this kind of service requires a large free base, and I’m not convinced people feel they get enough value from it that they’d be willing to support / pay. It’s sad, but it may just be time for me to keep moving forward with my other projects and wish Grazr.com and the widget a fond farewell.

I took this picture the other day while we were driving around downtown. I love the light, the mixture of old and new architecture as well as the perspective. I’m not generally a good photographer but I thought this one turned out pretty cool. (Oh, and it’s a pic with my new iPhone 4)

It’s tough to put my finger on exactly what it is. I think it comes down to one simple question: What is the objective of a startup (or business plan) competition? Is the objective to generate interest in entrepreneurship, to get more people talking about startups and interested in the process? Or is the goal to provide validation for startups and indicate, in some way, that some startups can be judged to be more likely to succeed than others.

It seems like the process for competing in business plan or startup competitions is only tangentially related to what really needs to be done to launch a real, viable startup. If your goal is to launch a company, which I assume is the goal of most of the participants, then I have a feeling that excelling in the environment of these kinds of competitions is, at best, inefficient use of time and effort and at worst, teaching potential entrepreneurs the wrong things to focus on for launching a company.

Of course, this is assuming that there isn’t a strong correlation between winning or placing in a startup competition and general success as a startup company. I wonder if the success rate of companies participating in startup competitions is any different from startups “in the wild” and if this data is available somewhere.

This unsettling feeling with regards to startup competitions was recently stirred up again after attending the Momentum Summit put on by Scott Kirsner. One of the guest speakers was the Chief Scientist at Akamai. He spoke about the startup history of that company and talked briefly about the MIT 100K competition (then called the 50K) at Sloan, a club program at MIT. I’d often heard Akamai trumpeted as one of the success stories of that particular competition. At one point he said something to the effect of “The common mythology is that we won the 100K competition, the truth is, we didn’t even place in the final round.”

Akamai is interesting from a couple of perspectives. The first, if we’re assuming that startup competitions are for generating interest and entrepreneurial activity, it was clear from the story that Akamai would never have happened if it wasn’t for the impulse from the 100K competition. On the other hand, one of the most successful companies to emerge from the 100K was eliminated, and didn’t even place in the competition. Granted that’s only a single sample, but does it maybe indicate that winning such a competition has no real correlation to success as a company? In that light, what does it mean to win a startup competition? If winning is relatively meaningless from the perspective of startup success, what’s the impetus to participate for people for whom that’s the real goal?

Startup competitions are popular because they overlay structure, clear rules for winning and losing over an inherently messy and unpredictable process. I think that’s why most schools and governments like startup competitions, they make the unpredictable predictable and establish a clear way to crown winners and losers. Startup success is, unfortunately, a very messy and unpredictable affair.

I’ve been trying to “hack the startup process” with a new kind of startup lab for a few months now. As part of the process of trying to launch TenZeroLab, I’ve been talking to a lot of people, pitching and describing the concepts behind it. Recently I started getting one particular piece of feedback: “Talk to Bill Warner, he’s got something going on you’ll be interested in”.

Finally, I managed to talk briefly with Bill at Tech Tuesday this past month and he suggested I join his “Anything Goes Lab” accelerator project happening at the CIC. Come Monday, I’m going to be participating in the two week pilot program.

What is it? It’s a co-working space but with a special focus on startups and teams. Bill and his partner Nick Tommarello have some interesting ideas about startups and startup formation. Many of the principles behind AGL seem to stem from Bill’s recent talks about intention and creating startups “from the heart”.

I’m not only excited to meet and work with some really interesting individuals, but also to see if I can contribute some of the ideas I’ve been working on with my own startup lab. I’ll be very interested to participate in the program, to see first hand the things that work in this environment and the things that don’t.

TenZeroLab and Anything Goes Lab appear to have some core principles in common, not the least of which being that we need to start helping each other. Facilitating the creation of trust-based working relationships is, I believe, a path to creating many more startups in the Boston area.

There’s a lot exciting stuff happening around the Boston startup scene and I’m glad to be a part of it.

I’ve dowloaded a coupe of iPhone games, but none have really grabbed me like “Tilt to Live” from One Man Left Studios.

The gameplay is really simple, you tilt the iphone to steer your little “arrow” ship to trigger weapons to destroy “the evil red dots”. You have one life, touch any red dot and it’s game over.

Overall it’s a very high-quality game. The graphics are simple but satisfying. The music is fun. There are little tongue-in-cheek “Awards” you unlock for various game events. When you get enough awards, you unlock new weapons. Even though it’s accelerometer-based, the control isn’t twitchy or annoying. The game has enough twists and strategy to keep me trying that just… one… more… time…

It’s available for $0.99 in the iTunes store.

Video from the One Man Left website:

When I was in the airport this week I had a minute before my flight boarded and decided to get a couple of magazines. On the stands I saw Startups Magazine published by Entrepreneur. Being a topic near and dear to my heart I decided to pick it up (didn’t have time to really flip through it).

The magazine was pretty thin and didn’t really have much meat to it. If you’re just curious about startups (haven’t yet put any plans into action or done any research at all) this magazine might be somewhat interesting to you. The articles were OK, but not very informative or particularly inspiring. Part of the magazine was devoted to tips for getting started but these tips should be fairly well known to anyone who’s been even remotely interested in startups.

I get the feeling that this magazine is an attempt to cash in on the sudden surge in the casual interest in startups. If you happen to see this on the stand and have more than a passing interest in startups, I’d recommend passing on it.