Entries Tagged 'django' ↓

Adrian Holovaty is the co-creator of Django, author of the Django book and is currently the BDFL of Django along with Jacob Kaplan-Moss. He studied journalism at University of Missouri–Columbia and has worked with many news sites including Lawrence Journal World and Washington Post. He currently works at EveryBlock, a startup he founded.

Shabda: Would you tell a little about yourself? You majored in Journalism, how did you move to Programming?

Adrian: Sure! I have a dual background in journalism and computer science. I intended to get a degree in journalism and a minor in computer science, but things got a little off track. I heard from a professional colleague that a certain Web site would probably hire me if I graduated a semester early, so I hurriedly dropped my CS minor and got special permission to graduate early — but once the time came to ask for that job, the company was in the middle of a hiring freeze and didn’t have a position for me. So, in hindsight, I should’ve stuck with the minor, but things ended up working out OK.

Because journalism and computer science don’t normally go together, I’ve had some success in this silly little niche of employing Web development in news organizations — “journalism via computer programming.” Professionally, I’ve always worked as a Web developer at news organizations, up until my current gig, which is to run EveryBlock. But in its own way, it’s a news organization as well.

Aside from all of that, I’m quite into music, particularly gypsy jazz, which is the music of Django Reinhardt (hence the name of the Web framework). I just recently attended a week-long gypsy jazz guitar camp in Massachusetts and had a tremendous time staying up until early hours of the morning jamming with great musicians from various parts of the world. If money were no object, I would love to play music full time. In the meantime, I’ll settle for posting YouTube videos of my guitar playing, for an audience that seems to be comprised mostly of 16-year-old boys who constantly pester me for transcriptions of my arrangements.

I live with my wife in the beautiful city of Chicago.

Shabda: What are your current roles and responsibilities at Django?

Adrian: Well, I co-created Django back in 2003 with my friend Simon Willison while we were working together. Now, along with Jacob Kaplan-Moss, I’m Benevolent Dictator For Life of the project — a title we shamelessly ripped off from Guido/Python. This means that I have a hand in all sorts of things, from high-level API design to checking in patches and fixing bugs. I’ve touched probably every bit of the framework over the years, including implementing the initial ORM (back when it was a code generator!), pair-programming the original Django template system and URLconf/view framework with Simon and building the original admin application with design help from Wilson Miner.

One of the things I enjoy the most is writing documentation — both because I like technical writing and because I cannot stand bad open-source documentation.

Shabda: Can you describe the philosophy behind Django. What are its overarching goals?

Adrian: A couple of years ago, I wrote the “Design philosophies” document, which sums up the main points nicely.

Generally, the goal is to make Web development fast, fun and easy for the developer, while keeping performance as fast as possible and code as easy to understand as possible.

Shabda: How did the move at WorldOnline from PHP to Python happen? Why did you start from scratch (which finally lead to creation of Django), instead of using something like Zope?

Adrian: Simon and I had been big fans of Mark Pilgrim (and still are!), and we’d read his online book “Dive Into Python.” This was around the same time that the PHP “infrastructure” I’d developed had begun to feel really, really crufty, so…one day we just decided to start using Python! It’s quite nice working at a small organization with a very loose management structure; our boss, Rob Curley, was cool enough to let the developers themselves decide which technologies to use, as long as the work got done. “I don’t care how the sausage is made,” he always used to say.

We looked at the existing Python Web frameworks/libraries, but none of them felt 100% right to us. Simon and I are both quite into best practices, such as clean URLs, proper use of GET vs. POST, etc., so we were very picky in our analysis of those existing tools.

Shabda: Can you tell us about Everyblock? Why did you choose to create your own mapping engine instead of using something like Google Maps? How hard has it been creating a new mapping engine?

Adrian: EveryBlock is the project I lead for my day job. It’s funded by a two-year grant from the Knight Foundation, and the goal is to experiment with block-specific news — that is, the site lets you enter an address (currently only in Chicago, New York City and San Francisco) and view recent news within a very tight radius of that address. I’m pretty confident it’s the most granular attempt at local news ever attempted on the Web. Nobody else is crazy enough to do it, I guess!

We do a TON of work collecting local information, normalizing it and pulling it together for people in one place. A large part of what we publish is government data such as crime reports, restaurant inspections, building permits, business licenses…and even local movie filmings. Much of this stuff either is buried in “deep Web” government databases or has never before been available online. A second part of what we do is detecting addresses and locations in news articles and blog entries. Plus, we pull in various other geographic Web stuff like Flickr photos and Yelp reviews.

All in all, our goal is to show you recent, geographically relevant stuff that you might not have heard about. Say the local newspaper wrote something about your neighborhood on page B-23 — would you really have noticed that article? Say there was an aggravated assault on your block — would you really have remembered to check your police department’s crime-reports Web site on a daily basis? That’s the basic philosophy.

Regarding our custom mapping engine…Let’s face it: Google Maps is so passe. As one of the original Google Maps mashup guys (I developed chicagocrime.org by reverse-engineering Google’s JavaScript, before the API was released), I have all the respect in the world for what Google has done to invigorate the world of Web maps. But it’s time to take the next step. The Google Maps API doesn’t give you any control over the colors of the map tiles, or change the fonts in street labels, or disable building footprints, or hide one-way street markers or subway stations or bus stops or any of the other stuff that’s essentially hard-coded in the map. So, at EveryBlock, we rolled our own maps so we could have much, much more fine-grained control over all of these things. Not to mention it’s a great way to differentiate ourselves from the 2.5 million boring, same-old-yellow-blue-orange Google Maps mashups out there.

Paul Smith from the EveryBlock team has written an article at A List Apart with more of the technical specifics.

Shabda: As per the Knight grant rules, you would be releasing the code for Everyblock next year. Would you not be giving away your secret sauce? How do you plan to maintain competitive advantage once that code is freely avaliable under a permissive license?

Adrian: Our competitive advantage is that we’re an incredible team, and I’m sure we’ll come up with a way to feed our families.

Shabda: What problems would you like Django to tackle after 1.0? What big features would you be most interested in having in Django, after it hits 1.0?

Adrian: Generally I’d like to add higher and higher levels of abstraction to the framework. The Django admin application is a good example of that – it’s not just an abstraction of an HTTP request; it’s an abstraction of an entire Web application! We should do more of those.

Shabda: What is one thing about Django which you absolutely love, and one thing which you think Django should do differently?

Adrian: I love the way URLconfs work — like a table of contents for your Web app. I also love template inheritance. I don’t love the fact that we’re generally slow in keeping up with tickets and feature requests.

Shabda: A lot of people these days have to evaluate between Django and ROR. How can they make this decision? When should Adrian Holovaty use ROR? When should DHH use Django?

Adrian: My answer is simple: Try out both frameworks and see which one you like better.

Shabda: Does Django need to market itself differently? What can Django community do for this?

Adrian: I’m comfortable with the amount of attention (or lack thereof, depending on your perspective) that our project gets. I’m comfortable with the size of the community. I’m comfortable with the fact that the right people have found out about it through word of mouth, books, blogs, Google App Engine or wherever else. If things continue at their current rate, we’ll continue to do just fine, as a healthy open-source project.

The one thing I’d ask the community to do is to continue staying civil, polite and approachable.

Shabda: What does the phrase ‘journalism via computer programming’ mean? How can these two divergent fields be tied together?

Adrian: “Journalism via computer programming,” in my opinion, is when a journalist writes code to tell a story. Instead of talking on the phone with sources to gather facts, this could involve screen-scraping Web sites to gather raw data. Instead of writing a newspaper article, this could involve building a database-driven Web site.

Journalism has several subdisciplines — photography, information graphics, video. I advocate that computer programming should be another one of those subdisciplines. Just as a newspaper employs photographers and graphic artists, it should employ programmers who help gather information and tell stories with it.

Shabda: How can traditional journalists do more ‘journalism via computer programming’? How can programmers do more ‘journalism via computer programming’? Is it easier for Programmers to move to this field, or for Journalists?

Adrian: I believe it’s easier for programmers to become journalists than it is for journalists to become programmers, but both sides need to gain an appreciation for the other in order for this sort of thing to happen more often. Fortunately, some news organizations are starting to hire developers with this in mind, and some geeks are realizing journalism is a great, (mostly) pure field that lets you improve the world through information.

One concrete thing programmers can do is to look for jobs at news organizations, which desperately need technical talent. Developers, you will be loved, you will be treated like geniuses, and your non-techie coworkers will be very easily impressed!

Another route programmers can take is to get training in journalism – in fact, Northwestern University’s journalism school is giving out full-tuition scholarships for programmers who want to learn journalism.

Shabda: Thanks Adrian.

This was Adrian’s interview, the last in django-interviews series. But we have a lot of interesting things coming up, so stay tuned.

Jacob Kaplan-Moss is the co-creator of Django along with Adrian Holovaty, as well as the author of the Django Book. He has been involved with Django since before it was called Django. He is currently employed at Whiskey Media where his job is hacking at Django. He blogs on Jacobian.org. He graciously agreed to be interviewed at the 42topics blog.

Shabda: Would you tell a little about yourself? How did you get started with Django? What other software/applications have you worked with. (Both OSS and otherwise)?

Jacob: So a bit about me: I grew up in Silicon Valley, so like many other geeks I got started with computers really early; I was programming professionally before I graduated high school. I didn’t start out doing web development; in college I worked on video surveillance systems for airports, harbors, marinas, and highways. That’s where I found Python: I rewrote a Java-based camera controller in Python and haven’t looked back since. I started doing web development pretty seriously when I moved to New York and took a job for a design firm there. The job was pretty terrible, and the technologies were worse: the good sites were PHP, and there was a bunch of WebLogic crap that was absolute hell to maintain.
So in 2004 I saw the job opening that Simon and Adrian posted on their blogs — and jumped on it. Web development in Python seemed like a dream job, and it really was. So that’s how I ended up in Kansas working for the local paper. At that point I suppose I was using Django, though we didn’t have a name for it yet: we just called it “The CMS”.
I’d been getting more into Open Source all along, though I’d barely contributed (I think I got a single line into Python at one point years ago — that was my largest contribution).

Shabda: What areas of Django have you worked most in? What are you current areas of focus?

Jacob: Well, at this point I’ve touched pretty much every part of Django at some point weather in the form of improvements, refactoring, or just small patches and bug fixes. I’m far from an expert in any particular area, though, so I’d say my main role is more holistic: I’m most concerned with making sure that Django “feels” correct and that APIs and conventions match across the framework.
Right now my current focus is documentation: I’ve been working on improving the structure and organization of the documentation so that it’s easier to find what you’re looking for. We’ve got something like 40,000 lines of documentation so organization and metadata is critical.
Now that I’m lucky enough to get to work on Django at work, I’ll probably be able to take on some more big tasks like that in the future.

Shabda: One of the most loved things about Django is its comprehensive documentation. What is the motivation behind refactoring this? What is going to be the new organization? How is this progressing?

Jacob: Thanks — I’ve always thought that Django’s documentation set us apart from most Open Source projects, and I’ve been really proud of what we’ve done there. However, over the past year or so as the size of the documentation (and Django itself) grew I think we’ve slipped from “outstanding” to merely “above average.” To us (the core maintainers and I) that’s not acceptable.
The best place to read up on the project and goals is in a post I made to Django-dev a couple of months ago. In a nutshell, I’m breaking up the documentation into smaller, more manageable chunks — the current DB API is almost sixty printed pages! I’m also separating more high-level how-to’s and topical guides from the detailed API references that get in the way when you’re just learning.
There’s some tool improvements going on under the hood that’ll make the documentation easier to write, edit, maintain, and contribute to; hopefully that’ll help decrease the number of undocumented features.
The work’s pretty close to done, actually. I was trying to finish before I went on vacation a month ago, but didn’t quite get there. So that means I need to roll in a month’s worth of documentation improvements that happened to the current docs while I was gone, but that shouldn’t take much time. You can expect to see the new docs rolled out online very soon.

Shabda: What does Whiskey Media, the startup you are currently associated with do? Is your role there developing Django, or are you associated with other day to day activities as well?

Jacob: I think I’ll have to be a bit coy and not say very much about Whiskey Media; we’d rather let our actions speak for themselves than try to build some sort of artificial hype. I’ll just say that we’re trying to solve some of the big problems in web publishing; you can probably see why Django’s the best technical bet for a company trying to be the on the cutting edge of content publishing.
Most of my time at Whiskey is devoted to working on Django — coding, but also community management, evangelism, and organization. I also have some internal responsibilities, of course, but those are more nebulous: mostly I just help out wherever I’m needed.

Shabda: As you said “we’d rather let our actions speak for themselves than try to build some sort of artificial hype”. Do you see Django taking a potential hit in marketing due to similar belief of Django community? Does Django need to market itself differently than it has been doing till now? What can Django community do for this?

Jacob: Huh, that’s a tricky one. Well, let me break down the question a bit and look at a few different angles.
So first, I’ve always been a bit uncomfortable with the idea — pervasive in Open Source — that this is some sort of popularity contest. People are always comparing traffic figures, or numbers of job postings, or numbers of Google results, or whatever. I think the idea’s supposed to be that if Project Foo has more users than Project Bar that Foo somehow “wins”. But I don’t see it that way at all.
I mean, I moved from New York City to Lawrence, KS. New York City has a population of, what, 8 million or so? Lawrence has a population of around 80,000. Does that mean New York City is “better”? To most of those 8 million, sure… but to me? Hell no.
As in most areas, there’s no accounting for taste: one of my good friends here in Lawrence is a die-hard Perl fan, and though it makes him a bit twisted it doesn’t mean that I’m somehow “better” than him — we just have different needs and different ways of thinking.
All that said, though, there is undeniably a value in popularity.
Only a tiny number of people who use a given piece of Open Source software become involved in the community, and only a tiny fraction of those contribute back to the project, and only a tiny fraction of those contributors become long-term committers. So more users translates directly to more contributors, and more contributors brings more “value” to the project.
(By “value” I’m referring to code, good ideas, etc.)
So as you can probably tell this is something we struggle with. On one hand I think Django’s great, and it’s in the project’s best interest to persuade others to give it a shot. But on the other hand there’s a huge amount of bullshitting that goes on where software is concerned, and I try my best to not add to the pile.
It’s hard to tell when you’ve crossed the line from evangelism to hype, and hype can be toxic. Witness the recent backlash against Ruby on Rails: do you think they’d get such violent vitriol if they’d been more modest in their promotion?

Shabda: Talking of ROR, a lot of people these days have to evaluate between Django and ROR. What questions should they ask themselves to answer this question? (Well apart from “do you know Python better or Ruby better?”). To make this more interesting when would YOU choose ROR over Django, if you knew both Ruby and Python equally well?

Jacob: So first let me back up a step — I’ll answer directly in a moment — and make a quick point about web development in general. The past couple-three years have seen a radical improvement in the quality of tools available to web developers. It really wasn’t that long ago that CGI represented the state-of-the-art in web development.
Today, though, there’s really some fantastic tools available. Rails, Django, Symfony, Pylons, TurboGears, Seaside… any of these tools represent a major improvement over the CGI/PHP model of web development. If you’re still writing web sites the way you did five years ago, you’re missing out.
When it comes to which of these tools to choose, of course, we’re back to that question of taste. Each tool comes from a different world, and has a different “attitude” towards web development. The cool part, though, is that most new web development tools emphasize a quick start as a key feature: you could probably evaluate a dozen web development frameworks in just a couple of days. So my best advice is to to try a few and see what “clicks”.
It’s probably important to also pay attention to the language that the framework uses. One of the real pleasures of writing Django apps is that you get to take advantage of the awesome Python community. I’ve got a set of libraries in Python that I can’t imagine developing web apps without; many of those libraries don’t have analogues in any other environment.
So if I knew Python and Ruby equally well — I don’t — I’d still probably lean towards Python, and towards Django. However, there’s one place I can think of where Rails is far superior: Rails runs on the JVM. This is a big deal: there’s any number of large corporate environments where the JVM is the only game in town. And obviously if I had to choose between Java and Ruby I’d choose Ruby!
I’ll mention, though, that Jython (Python on the JVM) is improving by leaps and bounds, and that getting Django working perfectly on the JVM is one of the Google Summer of Code projects the Jython team is sponsoring.

Shabda: Coming back to the marketing question.. For most of non hackers choosing the framework to use is a big question, and the decision they will make depend on various non-tech factors, such as availability of capable skilled people. You mentioned that you would choose Django over Rails due to the awesome Python libraries. For the long term survival and growth of Django, do you not think that an early capture of mindshare in developer community is important? For example I have pitched Django to a fair number of people, and I always have to start with “Django, what?” as compared to “Yeah, we are evaluating Rails for our requirements.”

Jacob: Sure, there’s definitely a value of having Django be familiar to your friendly local Pointy Haired Boss, so in that context a certain amount of advertising is important. There’s nothing worse than being forced by management to use the wrong tool for the job. Keep in mind, though, that this cuts both ways: I’d feel pretty unhappy if there was a team using Django because someone read about it in CTO Monthly. I’d agree that we could be doing more in terms of “brand awareness” or something, but all in all I’m pretty happy with the size and quality of Django’s community.

Shabda: There is not much information about the early days of Django, so a little about that. How did the move at WorldOnline from PHP to Python happen? Why did you create a new framework, instead of reusing something like Zope?

Jacob: I wasn’t yet at World Online when we moved from PHP to Python, but from what I understand it was a pretty typical change. Adrian and Simon got fed up with the pain and suffering wrought by PHP, and wanted something cleaner and — most importantly — something that would be easy to maintain. Python really shines here.
The main reason we ended up building our own framework was that we didn’t know we were building a framework. We just wanted to “build cool shit” and, over time, we built tools to help us do that. It wasn’t until we started showing it off that we realized we had something that could be used by other people.
This, by the way, is one of the reasons I think Django turned out so great. If you sit down one day and say, “I’m going to develop a framework!” you’re almost certainly going to become an Architecture Astronaut, and if you ever actually finish the thing’ll be so over designed nobody will want to use it. If, on the other hand, you simply try to solve real-world problems in a clean, obvious way, you’ll eventually end up with a great tool.
Look at Rails, TurboGears, even PHP; they all started as simple libraries written by frustrated programmers just trying to get the job done.

Shabda: What are the overarching goals of Django? Again to make this more interesting, Here is a quote from you. “My work as a core developer of Django focuses on giving anyone — even (especially) non-programmers — the tools to create dynamic, content-driven websites.” Should not that be the job of something like Wordpress, while Django should aim to give “programmers, but not non-programmers — the tools to create dynamic, content-driven websites.”

Jacob: Wordpress is great if you want to publish a blog, but what if you want a website to track a book collection, or sell tickets to concerts, or organize a local farmer’s market? There’s a great deal that Wordpress (and other single-purpose tools) can do, but the amazing thing about the web is just how wide-open the possibilities are. There’s a whole world of possibilities out there, and the end-goal is to help anyone self-publish anything they can dream up.
One of the most fascinating aspects of the history of communication is how intertwined literacy is with social controls. For most of the history of humankind, literacy was strictly available to the elite. The Web created the greatest democratization of publishing ability in history, and has almost immediately turned into a battleground between traditional, centralized publishing and decentralized democratic publishing.
As a programmer, we don’t personally play much of a role in this sea change, but I do see the lowering of the barrier to self-publishing as something we ought to continuously think about.

Shabda: If Django’s goal is “and the end-goal is to help anyone self-publish “, does not that mean Django is trying to fill the niche of an Extensible CMS like Drupal, as compared to filling the gap left by PHP? Are non programmers really using Django, or are they using apps built with Django?

Jacob: No, you’re right that Django’s lower-level than something like Drupal. Django’s not trying to be a CMS but to be a tool you could use to build your own CMS. It’s easier to design your own content models than to shoehorn your publishing into a CMS limited by the ideas of its developers.
There are indeed quite a few non-programmers using Django, though in the long run I think the interesting trend is that more and more computer users are learning a little bit of programming — enough to develop a site with Django, say.

Shabda: (Since I ask this to everyone!) What is one thing about Django which you absolutely love, and one thing which you think Django should do differently?

Jacob: I only get to choose one thing I love?
I think my favorite bit of Django is the URLconf system. I love that Django forces me to think about URL design as part of my application instead of some byproduct; I’ve always hated web tools that try to someone pretend that you’re writing a desktop app. I’ll admit to obsession over my URL design from time-to-time, but I really enjoy clean, semantic URLs.
As for something we should do differently: the assumption that you’ll only use a single database is a bad one, and needs to be fixed. Unfortunately, it’s an assumption that we made really early on, which means that fixing it is going to be tricky. There’s some smart people working on it right now so I’ve got high hopes, but I wish we’d not made that assumption to begin with.

Shabda: Django started as a in house project and was later open sourced. At your current startup your major job is with Django itself. The Django book which you wrote was released under a permissive license. How difficult is it to convince people outside the hacker culture of the business value of open source? What are the best ways to do this? How difficult was doing this at World online?

Jacob: These days, thankfully, the business value of Open Source is pretty well-established. There really hasn’t been a lot of “convincing” necessary. For example, Apress didn’t just agree to release the Django Book under a permissive license: they actively encouraged it. You’d have to ask them if it was “worth it” in terms of sales, but I’m sure that being the first Google hit for “django book” didn’t hurt!
Releasing Django at World Online is actually an interesting story. We decided at the 2005 PyCon that we should Open Source some of our software, and started building a business case for doing so. We prepared a series of arguments — open source will increase our visibility and lead to easier sales and easier hiring, open source will improve the quality of our software, etc. — and took them to our management. All of those things turnned out to be true, by the way, but o our surprise, the argument that was the most effective was actually a “moral” one. We talked about how Open Source had helped our business (Apache, Linux, Python, etc.) and argued that it was time to “give back” to the community. The World Company has always been a company that’s tried to be a conscientious part of our local community here in Lawrence, and they really jumped at the chance to participate in the global Open Source community.
I like to tell this story, by the way, because it really makes me hopeful that this “hacker culture” is in fact compatible with business culture. Fact is that most businesses are in fact concerned with doing the “right thing” — they just often don’t know what that is when it comes to technology. Of course the business case for Open Source needs to be there, too — and it is — but I think there are a lot of companies that’ll jump on the chance to “give back.”

Shabda: Before we close would you like to share any tips with us? Jacob: If you don’t read the Django community aggregator you really should: there are some incredibly smart people blogging about Django and you’ll learn something new from all of them.

This was the interview of Jacob Kaplan-Moss. We have a few more Django interviews coming, before we close this series of Django interviews, so stay tuned.

Michael Trier is a long time Django user and evangelist. He has worked with a number of technologies including Rails and .net. His insights on marketing Django to traditionally Enterprisy areas were extremely informative. He produces TWiD, along with Brian Rosner which is great to keep abreast of the latest happenings in the Django community. He graciously agreed to be interviewed by the 42topics blog.

Shabda: Would you tell a little about yourself, how did you get started with Django, what other projects have you used or are associated with?

Michael: Well, I’ve been programming ever since I can remember, probably around 11 years old. I grew up in Silicon Valley, and that whole story is a pretty interesting one. I did the usual thing of starting out with languages like ASM, C, C++ , Pascal. Moved on to things like Delphi, VB, and most recently I spent quite a bit of time with Ruby, Rails, and within the past year and a half dabbling with Django.
I came to Django for a particular reason. I was focussed on building a high content push type of site and it just seemed like Django was a much better fit for that than Rails. Obviously I could have done it with either language, but I believe in using the right tool for the job.

Shabda: You are using Django for your next venture. What are the specific areas where using Django been a better way to develop for you, compared to any other choice you might have made, say Rails?

Michael: I hate to bring up the scaling issue, but that was certainly something I was seeing as a problem with Rails. The type of site I’m building I hope to see some pretty high traffic (don’t we all). So that was one thing. The other thing was that it seems like if you’re doing a large content driven site, Django just makes that very easy. Additionally, as most people say, one big win for Django is the built in Admin. It has allowed me to focus on the front end of the site, while giving the other people involved a way to immediately start working on the content part. Right away this helps us see where we may need to enhance functionality or perhaps we’ve built in too much flexibility.
That kind of feedback comes back to us quickly and so that is invaluable.

Shabda: How do you compare Rails to Django? In what areas is Django better than Rails (Apart from scaling/efficiency)? What does Django still need to learn from Rails?

Michael: That’s a good question and not something I’ve given a lot of thought to, but off the top of my head I would say that with something like the Admin, Django definitely wins over Rails. Now within the Rails community there are a lot of interesting third-party plugins that have attempted to mimic the Django admin, but up until now those that I’ve looked at have fallen short. It’s a huge effort as we’ve seen with the amount of work being put into the NewForms-Admin rewrite.
I also think the middleware stuff in Django is very nicely done, although it could use a bit more options in terms of request / response ordering of the middleware items. I also think Django templates are just perfect. I really can’t see how to enhance on those at all.
As far as Rails is concerned, there’s a lot of nice features in rails that Django could learn from, and most of it just has to do with the maturity of the two projects.
Rails caching takes caching one step further. Rails has model level validations which are very nice and quite frankly the right way to do it, in my opinion.
ActiveRecord, Rails’ ORM, has things like aggregation support, and supports a lot of flexibility in how you are able to filter the dependency relationships between your models with things like has_many :through (basically intermediate models).
I also like the before_filter and after_filter type of stuff in rails. It makes it real easy to invalidate cache or do other things in an event driven way.
Oh one thing back on the Django side, I think NewForms are done really well. It’s an elegant solution. And it is interesting because in the .net world I’m starting to see a copy of that through things like dynamic data controls.

Shabda: One specific area which I believe Django can learn from Rails is marketing. What would you say to this? With 1.0 release coming soon, how can Django start to market itself better.

Michael: I would agree with you on this. I think the impending 1.0 release will definitely do a lot on its own to market the product. There are also at least 3 more books coming out in the next several months. The Google App Engine announcement that heavily featured the name of Django has also helped to bring it focus.
But there’s another element that I don’t know if you can fabricate and that is that DHH is a charismatic individual.
In terms of real things that can be done are obviously things like more screencasts just featuring the product. It’s interesting to note that consistently the number 1 screencast on iShowU is the Django one that features a simple tutorial on Django. Although a lot of people have no interest in screencasts, there are a lot of people that do. We all learn in different ways.
I also think that on the djangoproject.com weblog we can do a much better job of regular blogging on what is going on within the community.
There are times when you look at the weblog and it’s 2 or 3 months old. From the face of the website you would think that nothing is going. The last release is over a year old. Meanwhile those of us that are in the community every day, we see that tons of stuff is happening. This needs to be communicated better.

Shabda: Talking of GAE, what effect do you see of it on Django? How can Django make the GAE-Django integration painless, or even if this should be done, and what efforts the Django community should expend on this instead of say focusing the efforts on 1.0 release?

Michael: I think GAE helps Django, but only a little bit. It helps it from the standpoint of name recognition and will cause a lot of people to say to themselves, “what’s this Django thing all about.” I think GAE really helps Python in a real way.
In terms of making the integration painless, that’s going to be a very difficult task. I have a taste of that with the django-sqlalchemy project I’m working on, and I’m just mapping a relational model to another relational model. With GAE it’s quite different. That said, Google folks are working on on a project to do “some” mapping of Django to GAE. I think over time it will expand in its focus. I think if they are willing they are the best people to do the work. I’d rather see Django focus on getting to that 1.0 point.
As far as GAE itself is concerned, I’m kind of on the fence on the real value there. I know that people like Jaiku are porting their stuff over to it, and Kevin Rose said that it would have been a great platform for something like Digg. Personally, I’m not really convinced of that, but it’s still early so we’ll all have to wait and see what comes of it.

Shabda: Your TWiD has been a great help for people to keep track of all the happenings in the community. Would you like to share some interesting tidbits you have learned with TWiD, and point to the more interesting ones?

Michael: Thank you, I’m glad that people find it helpful. I think one of the interesting things is the amount of work that goes into it. A lot of people are surprised because neither Brian nor I are very professional, so when we’re on the mic it sounds like we’re just sitting there talking about whatever. The reality is that it takes quite a bit of work in finding the stuff we want to discuss, attempting to understand enough about the topics to at least sound somewhat intelligent on them, and then there is the recording and post-production.
As far as actual topics the two recent episodes on Internationalisation have been a great learning experience for me personally. Going into it, I really didn’t even know enough about it to know what questions to ask. Thankfully Malcolm Tredinnick put the whole thing together. He’s been a huge help to the show.
Another show that was fascinating for me was the one on GeoDjango with Matthew Wensing. It’s a fascinating topic and I hope to put together more shows on that subject.
I think a lot of people like the interviews, and we do too, but we also don’t want to make the show just interviews every week. So we try to split it up.

Shabda: Would you tell a little about the venture you are working on?

Michael: Sure. The project is a hyper-local media site. We’re working on providing an online place for a mix of general media type of content (stories, events, etc…) with citizen journalism types of offerings. A lot of people are working in this space, trying to figure out how you bring the big media type of stuff down to a local level.
We then want to mix some of that with interesting datasets, a la EveryBlock, as well as provide some level of social interaction. The cool thing is that once we release we’re going to make the entire thing available on a New BSD license.
Frankly, initially it’s not going to be that interesting. A lot of what we’ll provide in the way of Classifieds, Marketplace, Aggregators, stories, etc.. will be similar to the types of offerings that you see in things like Ellington or just about any newspaper’s online presence these days. Down the road I hope that we can expand it into something quite useful.
The framework we’ll be releasing is called ArEyah. So expect to see something from me on that in the next several months. Timeframes are tough to nail down at this point because this is a side project in addition to some of the other things I’m involved in.

Shabda: For many people, such as me, the question is what business advantage remains for you after you are releasing all your secret sauce under an open license. For example even EveryBlock will have to release all their source under an open license, after the end of the Knight Grant expiry period. What would you say to this? What do you hope to achieve with releasing the project under a BSD license?

Michael: That’s really a good question. For me it was two-fold. First I wanted to take the “intellectual value” of it off the table. In other words, I did not want to be in a position with my partners where I had an extremely unfair advantage. Secondly I really think this is something that could benefit communities everywhere; of course that remains to be seen. If that is the case, then I think a lot of individuals would get involved and make it a better product for the benefit of all.
As far as competitive advantage, I really see that being in the execution. In comes down to how well we serve our community. If we are not doing a good job of it then someone else should be able to come along and beat us out there. There were certainly be some things that are specific to the community that we’re targeting that don’t have a place in the general framework. Those things will be our own and they will be highly tailored for our use, in order to serve a specific need we have here.

Shabda: We see a lot of comparison going on between Django and ROR, or Django and Turbogears. But we do not see enough comparison between Django and other traditionally ‘Entrprisy’ frameworks. Say Java based frameworks like Struts+Hibername or Asp.net. As you have worked with .net, how would you compare Django with these frameworks. What can Django do to make itself popular in the areas dominated by these frameworks?

Michael: The differences have less to do with feature set comparisons. A lot of what I do every day for corporate clients could be done much more quickly and actually often more robustly with something like Django. So it’s not a thing where you can say feature x is in .net but not in Django. That said like any framework / language there are edge case types of things where you might say “Java is the right tool for this job.” What it really comes down to is the corporate culture. .NET and Java own those corporate cultures. There are real reasons why it makes good business sense to build your corporate infrastructure on Microsoft products. You can pick up the phone and get three .NET developers tomorrow. Regardless of whether your company is based in Louisville, Kentucky or White Plains, New York.
In the case with Microsoft, much more than in the Java world, they provide a singular full-stack solution. No one, in the executive sense, needs to make any more decisions about which reporting tool, which database backend, or which IDE they are going to decide to use. Often that alone is motivation enough.

Shabda: Your last post on your blog was about the benefits of DVCS as compared to Centralized systems. What are the compelling benefits of DVCS over, say, SVN. In particular how can moving to a DVCS help Django?

Michael: Well I’m somewhat new to the DVCS world, as are a lot of people, so I’m no expert on the subject. To me though the benefits of a DVCS are at more of a personal level. I have seen tremendous personal benefit in being able to commit while sitting in a coffee shop, or being able to branch code locally and then merge that back in.
As far as benefit to a centralized project like Django, I guess it remains to be seen. I’ve been watching the shift of Rails to Git with great interest.
So I guess in summary, I just don’t know enough at this point to say that it would be beneficial. The shift is probably more psychological. I think, for a lot of projects, it would be just another way to have a centralized repository.

Shabda: You and Brian Rosner are working on django-sqlalchemy. How would a sqlalchemy based ORM be better than the current implementation? What is the status of this project?

Michael: Yes, Brian and I are probably the primary committers, but we have several other individuals that have pitched in on the project. I like to think about the benefits in two separate areas. First, there’s the approach where you just plug django-sqlalchemy in and continue to use Django with its filter syntax, etc… In that case the benefits you gain are not benefits at the ORM level (because you’re still using Django’s syntax, which doesn’t support things like aggregation). The benefits instead are things provided as a result of having SQLAlchemy as the backend. So this would be things like multi-database backends, sharding, additional database support like DB2 or Firebird.
The second approach is where you actually need things like aggregation or more complex queries, without resorting to raw sql. In that case we expose SQLAclhemy’s ORM right on your models. So the full power of SQLAlchemey gets exposed and available for you to use whenever you need.
So I could technically have 90% of my ORM code just using Django’s syntax but then realize that I need to do something a little outside of its capabilities. In that situation I might chose to just use the exposed properties to get what I need.
Finally there’s one more thing related to this that we still do not have clear at this point, but will come down the road, and that’s actually adding different functionality at the model level. This might be things like Intermediate Model support being made available through django-sqlalchemy.
As far as status of the project, it has been moving along very well. We have a few more filters to implement and a handful of management commands, plus lots more testing. One of the things I’ve done in the past week is to code up a test application using django-sqlalchemy to see in a “real-world” sense where some of the problems are. That has been really helpful.

Shabda: So if the extra features added by SQLachemy is not needed, then this aims to be backwards compatible with current Django syntax?

Michael: Yes, our aim is that you could plug it in and run all your stuff just as is. In other words with django-sqlalchemy as your backend db (that’s how it gets exposed) we should be able to pass all of Django’s test. Once we’re able to do that we tag it 1.0 and get some people hammering on it.
We do have one big hurdle that I have not discussed. Currently we are using multiple inheritance to modify the Django classes. That doesn’t work for the contrib apps or third-party apps because that would require a change to their code base.
We took this approach originally just because we wanted to focus on the mapping issues first and prove the concept. The eventual plan is to use some class replacement techniques to inject our stuff right into the Django models at evaluation time. That will make it work across the board.

Shabda: Before we leave. Would you like to share a tip, or hard to find information about Django with our readers?

Michael: Not really a code tip, because we do those each week on TWiD, but more of an approach tip. The generic views stuff is extremely powerful, and quite often I see a lot of new users doing stuff in views that could easily be done with a generic view, or with a wrapped generic view. James Bennett has a great post on this, and I suggest everyone check it out. I think often, especially for people that are new to the community, the power of it is overlooked.
Finally one more thing, spend some time in IRC. IRC is a great way to get an education in Django very quickly. Reading the questions and the responses has been invaluable to me in learning how to use Django more effectively. It’s a great community.

Shabda: Thanks a ton for this great interview. It was extremely informative and interesting.

Michael: Thank you.

This was the interview of Michael Trier. This week I am not going to any more interview, but stay tuned for next week, we have even more Django interviews coming.

And of course, the 42topics is live now. And we have a Django section. (How 42topics works?) So join now, and lets get rolling.

For all its technical merits, Django is still a very niche technology. It is my belief that the thing which is holding Django back the most, is due to one of its strengths.

Making reusable apps is easy and simple in Django. In Django this is the correct way to do things. You take a few apps, mix them together in your project, and deploy to start your site.

Compare the installation steps of Wordpress and an imaginary blog software better than Wordpress called Djangopress.

Wordpress

1. FTP wordpress to webserver.
2. Point browser to site.com/blog
3. Next-Next-Next done.

Djangopress

1. Svn checkout Djangopress
2. Svn checkout django-registration
3. Svn checkout other Django apps Djangopress depends on. Maybe django-mptt, django-threadedcomments or a few others.
4. Edit your settings.py to add all these apps to INSTALLED_APPS.
5. Add database settings, and other changes if needed.
6. Telnet to your server and do syncdb
7. Create templates. Done.

This does not take into account the extra hoops Apache makes you jump through, compared to using a PHP app.

How I got started with web programming.

I wanted to run a forum. PhpBB was free, and seemed most widely used. Installed it, and wanted to tinker with it, so learnt Php. If there was a different forum software, which was technically superior, but which asked me to write templates for it before I could start a forum, guess which one I would have chosen?

So how to popularize Django.

In my interview of James Bennett, I asked what is Django’s killer app. And he said there need not be a Killer app for Django, reusable apps will do. I guess I will have to disagree. Even internet needed a killer app to get breakthrough popularity. Let’s see what a Killer app gives you.

1. It fills a big niche, so people are forced to learn your language/framework.
2. It forces the Hosting company to support your language/framework.
3. If a large number of places use it, it gives your framework name recognition.

So to popularize Django, I propose setting up DjangoPackagedApps.com to distribute packaged Django apps, to complement reusable Django apps. A packaged Django app, must have these properties. 1. All dependencies must be included. 2. Beautiful templates must be included out of the box. 3. Users must not need to modify anything in settings.py apart from the database settings.

And installing the PackagedApp must be no more than the number of steps needed in Wordpress.

1. Svn checkout/FTP DjangoPackagedApp
2. Only thing to edit in settings.py is database settings.
3. Do syncdb. done.

Do yo use Django? Do you program? Find things which YOU will love reading at 42topics.com.

Russell Keith-Magee is a longtime core contributor to Django. He has worked extensively with the Django testing and serialization components. He is currently working on django-evolution, the ability to do schema evolution from Django, an often requested feature, and is mentoring a GSOC proposal to add aggregate queries support to Django ORM. He is currently a Senior R&D Software Engineer at plugger.com.au. He graciously agreed to an email interview with the 42topics blog.

Shabda: Would you tell a little about yourself? How did you get started with Django? What areas of Django have you contributed to?

Russell: I’m a thirty-one year old, father of one, living in the most isolated capital city in the world - Perth, Western Australia. Strangely, for all my involvement in Django, my background has almost nothing to do with the web. I studied Physics as an undergraduate, and studied neural networks for my PhD. My first job was with a startup in the defence industry developing simulation frameworks. Over time, mostly as a result of my association with Django, I’ve become more involved in building web services.

Shabda: How did you get started with Django? What areas of Django have you contributed to?

Russell: I’ve always had a bunch of little pet projects that I’ve fiddled with in my spare time. Over time, these projects have migrated over a range of languages until I found (and fell in love with) Python. Around the time of Web 1.0, I started looking at various web frameworks (PHP, Zope, etc), but I never really managed to get mental traction - they were either so complex that it was hard to work out where to start, or they provided a bunch of tools but didn’t really make it obvious how the pieces all fitted together. Plus, the documentation for these frameworks was up to the common open source standard - an even mix of the awful, the incomprehensible and the non-existent.
Around mid 2005, the buzz about Ruby on Rails started to grow, so I took a look. I could see that it had promise - it was an interesting new way of looking at solving web problems - but again it suffered from the documentation problem. One day, I found a blog entry referring to Django, and finally all the pieces fell into place. The documentation was excellent, the simple stuff was painfully simple, and there was a clear separation of concerns - there is the URL module which routes requests, an ORM to access the database, etc.
At the time, the pet project I was using to test new frameworks was a time/task tracking system. After working through the tutorials, I tried implementing a task tracker, and quickly realized that Django didn’t have support for aggregate functions. I made a few suggestions, which were universally rejected (in retrospect, for very good reasons), so I went digging in the code to try and come up with some better ideas.
In the process, I ended up spending a lot of time in the query system, which put me in a position to submit patches for a few small bugs in queries. Once I had found my feet, I tackled some larger enhancements. One of my first big contributions was modifying the filter syntax so that queries could span reverse relations; that is, if Book has a foreign key on Author, I modified the syntax to allow Author queries to ask about their related books. Of course, between the Magic Removal changes and Malcolm’s Queryset refactor, most of this code has now been thrown to the wind, but at the time it was a big improvement. Adrian noticed the contributions I was making, and offered to give me commit access and make me a core developer.
Not long after I joined as a core developer, the Magic Removal branch started, which consumed pretty much all my efforts for about six months. This involved more changes to the query system, but also with the way models were defined and instantiated. Following that, I developed the test framework, which also involved working with the serialization framework. I did some work on newforms-admin in the early days, including developing the new Media definition framework and a refactor of form input fields for files. I have also spent some time on side projects that aren’t directly targeting the Django trunk, but might be integrated one day. The most notable of these is Django Evolution - an implementation of schema evolution for Django.
I’ve been a little bit quiet over the last few months; I’ve just changed jobs, which is consuming a lot of my time, and I’ve had a few bouts of illness which have slowed my progress. I have also been helping out James Bennett by acting as Technical Reviewer for his upcoming book “Practical Django Projects“; there won’t be much to show for this until the book is published, but when it comes out, you’re all in for a treat - it’s going to be a great resource. I’m hoping to get back into the swing of things in the next few months. One of the first new tasks will be a very old issue - I’m mentoring a student (Nicolas Lara) in a GSOC project to implement aggregates in the query system. So - after almost 3 years, my original itch may actually get scratched.

Shabda: Have you worked with other competing frameworks, say Ruby On Rails, Turbogears etc?

Russell:
I’ve looked briefly at a lot of other web frameworks. In the early days I looked at Zope and PHP while gaining a foothold in the web world. Since joining the Django project, I occasionally look at what the other frameworks - in particular, Ruby on Rails and TurboGears - are doing. It’s occasionally helpful to see how other people are (or aren’t) solving various problems, borrowing the good ideas where possible.

Shabda: What other software/applications have you worked with. (Both OSS and otherwise)?

Russell: A little bit of everything. I used Linux as a personal computing platform back in the glory days of Slackware and Yggdrasil. I was compiling KDE and GNOME before they were even v1.0, and I’ve toyed around with writing for both frameworks. I’ve used Windows in a commercial environment, and I’ve gotten to know the eccentricities of every version of Microsoft Visual Studio from v6 to 2008 in far too much detail. I’ve written Java, both server side and client side. I’ve spent a lot of time with a simulation protocol called HLA. One of the more notable products of that work was securing some funding and commercial support for an open source implementation of the HLA protocol. I’m a huge fan of Linux, especially on the server side. However, I attribute most of my recent productivity to being a Mac Switcher. I found that once I made the switch, I spent a lot less time messing around with compiling the latest, slightly less buggy version of some esoteric utility, and a lot more time building useful code. It’s possibly a little ‘post hoc ergo propter hoc‘, but my Mac switch correlates almost exactly with being accepted as a Django core developer. Regardless or the causation, give me a MacBook Pro, a copy of TextMate and a good net connection and I’ll be a very happy man for a long time.

Shabda: You have worked with a number of web frameworks. What made you choose Django over them? What are the strong points of Django compared to say ROR? Compared to Turbogears? What can Django learn from these other frameworks?

Russell: I haven’t worked extensively with any other web framework. I’ve had the occasional poke around - mostly to see how the other guys do things. To my eyes, at a technical level the various modern web frameworks have a lot more in common than they have differences. All of this generation of web frameworks are about providing a clear layer between the database and the user, proving a simple mechanism for mapping web requests to functions that serve those requests, and providing a clean way to render content for the responses. Sure, there differences in language of implementation. There are some philosophical differences like whether templates should allow executable code. There are also some feature differences. However, these are all minor in the grand scheme of things. The philosophical differences haven’t stopped anyone from building web applications in any of the frameworks. Given enough time and resources, any feature could be implemented in any framework. Language choice is also largely irrelevant. I’ve used a bit of Ruby, and while it has some interesting features as a language, the difference between Python and Ruby is a lot less than the difference between, say, Python and C++. What this leaves is the stuff outside the technical realm - resources, documentation, and the community. This is the reason I choice Django in the first place, and I still think that they are areas of strength - I certainly don’t regret my original decision. I remember when I first found the Django website - I very quickly got the impression that it was built by a group of people that really cared about what they were building, and how it was perceived. The website and other resources weren’t just rapid constructions or reflections of this week’s design fad - they were designed with care. The same went for the documentation - it wasn’t just a bunch of documents autogenerated from source code comments or a sprawling wiki - they were custom written by someone who knew how to write, and more importantly, knew how to edit. As a result, Django’s tutorials and documentation made getting started an almost painless process - both as a user, and as a developer. The attitude of the community is also important; Adrian and Jacob have established a very civil community and the community as a whole tries to keep it that way.
I think the biggest lessons Django can learn from other frameworks is in how to market itself. Because we’re still pre-v1.0, we haven’t really been aggressive about marketing ourselves. Some of the other frameworks have been selling themselves for a long time, so we are in a position to learn from their experience.

Shabda: One of the most requested features with Django is schema-evolution, which you are working on. What are the design goals for django-evolution? How production ready is this?

Russell: At the moment, Django Evolution handles the simple cases (add/delete/rename simple field) fairly well. There are some bugs in the more complex cases, particularly those involving foreign keys and many-to-many fields. It tends to work quite well for SQLite and Postgres; MySQL tends to be more problematic. At this time, I’d classify it as “production ready for capable developer”. As long as your schema evolution needs aren’t too advanced and you’re willing to do a bit of hand holding and carefully check the SQL that is produced, you can use Django Evolution right now. If you’re not as confident with raw SQL, you might want to wait a while. Obviously, the long term goal is to make it “production ready for everyone”, and support every possible change that you can make to a Django model. Completely aside from fixing bugs and supporting other databases, one of the big open issues is populating data into new columns. At present, we have some simple mechanisms to put in default data, but these are somewhat limited. Longer term, we’re aiming to be able to populate a new column with data from existing columns. This will require providing a subquery as initial data, and my hope is that the new Queryset Refactor features will make this sort of thing much easier.

Shabda: I asked this to Malcolm as well, but he was unwilling to make a prediction. So let me ask this slightly differently. What problems would you like Django to tackle after 1.0? What big features would you be most interested in having in Django, after it hits 1.0?

Russell: My efforts in prognostication have a nasty habit of going badly wrong… but here goes:
One of the recent shifts in in the Django project as a whole has been the move to allow user extension as much as possible. The ability to have user-defined field types, database backends, and management commands are all examples of the push to move Django development into the community as much as possible. I would be very happy to see the majority of Django v2.0 features be a merge of established community-based projects into the trunk, rather than projects tightly integrated with Django internals.
That said, some changes can’t be done as extensions. One such reoccurring feature request is for multi-database support. It isn’t something I have a huge need for myself, but it is a reoccurring request on the Django users mailing list, so it would be good to be able to support it.
Support for aggregation functions (sum, average, etc) in the ORM is a feature I have a use for, and I’d like to see - especially since the lack of aggregation functions is the reason I got involved with Django in the first place. I’m currently mentoring a GSOC project to implement this capability, but I expect that this won’t be complete until post v1.0.
I would also expect that the resources around the community will get some attention. There is plenty of room for new tutorials. There are also a few contrib services that need much better documentation (in some cases, any documentation). Given that v1.0 is a point of API stability, it also makes a good time to add lots of new tutorials and other documentation. Formalization and documentation of some of Django’s more notable internals (like the _meta model class and the various extension mechanisms) would also be nice to have.

Shabda: You have worked in many different areas of Django. What suggestion would you give to somebody who is just looking to start contributing to Django? Are there some areas, where the learning curve is flatter, and people can make meaningful contribution more easily?

Russell: My suggestion would be “don’t be timid - jump in”. Even the most complex parts of Django - like the query system - are relatively straightforward once you’ve spent a bit of time walking through them. If someone is looking to contribute to Django, my suggestion would be to do the same thing I did when I was starting out - pick a Django component with a bunch of bugs, pick one, write a test case for the bug (if there isn’t one already), then walk through the code until you can fix it. Rinse and repeat, but stay in the same component. Eventually, you’ll know that component as well as anyone in the world, and you’ll be able to work on more complex bugs or contribute a feature. Occasionally, tracking down one bug will lead you into another component; use that as an opportunity to learn how another part of Django works.
The other thing to keep in mind is that working on Django means more than just code - if you want your contributions to be taken seriously, you need to get in the habit of writing rigorous tests and comprehensive documentation. Don’t just fix the bug and then complain that we don’t commit your tickets - you need to do the whole job.

Shabda: A lot of Django people come from diverse backgrounds, You have studied Physics, Adrian studied journalism, Malcolm mathematics and James philosophy. Is there something about with Django which attracts people from diverse backgrounds (keep simple things simple …)? Has a lack of people with CS degree been a problem sometime, as in “We wish there was a CS guy to discuss this problem with”?

Russell: My undergraduate degree is in Physics, but my honours and PhD program were both in pure CS. Plus, for all their lack of CS degrees, Adrian, Malcolm and James have certainly proven they know how to cut a line of code, and there are plenty of people in the Django community with pure CS backgrounds that contribute ideas and code. I don’t think Django is suffering from a lack of CS experience.
However, I do think that having a multidisciplinary background does predispose you to a certain way of looking at the world. It has been my experience that some people with a narrow IT/CS focus lose sight of the fact that at the end of the day, a computer is a tool, most people don’t like them very much, and only put up with them to the extent that they help to make their lives better. I can appreciate the elegance of a neat algorithm as much as the next CS major, but at the back of my mind I always have the question “How can I use this to solve a real world problem?”
One of the reasons that I suspect Django has been a success is that it has never lost sight of the fact that it exists to solve problems. Internally, it does a lot of smart things, but outwardly it tries to be as simple as possible. You download the code, and it works - you don’t need to resolve dozens of dependencies. The development server just works. There is a simple and easily understood pipeline from request to rendering. Django identifies that building web applications is a problem that people need to solve, and tries to provide the simplest possible path to let that happen.

Shabda: You have worked with a defense startup, and defense industry is presumed to be very conservative. What can Django do to market itself to market itself to people outside web 2.0 niches, such as in large corporations or defense related software?

Russell: There is a running joke in the Australian military that the Army’s idea of the perfect technology advancement would be a self-sharpening grease pencil. It’s not quite that bad, but the military is very conservative. This is understandable, because lives are potentially on the line if a technology fails. However, it does breed some interesting limitations on new technology: in the circles I was working, IE6 on Windows 2000 was considered to be a very modern web platform.
The way to market to people outside of Web 2.0 niches is to stop trying to sell Web 2.0 as if it was an answer to anything. Web 2.0 is a technology. Web 2.0 may give you a competitive advantage. However, your customer doesn’t give a pair of fetid dingo’s kidneys about your competitive advantage. The way to sell your Web 2.0 idea is to forget all about how cool the technology is, or how great it is that you can do this as a RESTful web service. Instead, focus on how your solution solves a specific problem of your customer in a way that your customer is comfortable with.
A person in industry has a problem. They need a solution. They don’t particularly care if the solution is implemented using an alien slave hiding in a cardboard box connected to the server using electric spaghetti - as long as the solution is cheap, reliable, solves their problem, and doesn’t require them to fundamentally change the way they do business. If you can convince your customer that your product meets these criteria, you’ll have a long and profitable career in the military-industrial complex - regardless of whether you use Web 2.0 or not.

Shabda: What is one thing about Django which you absolutely love, and one thing which you think Django should do differently?

Russell: On a technical level: I think the decision to keep logic out of templates is 100% correct. Keeping logic out of templates forces the separation between content and presentation, and simplifies the interface for the design team, who often will not be gung-ho programmers. I think this is one of the best decisions that Django has made - I know there are many in the world that claim that Django’s templates are it’s biggest weakness, but I simply don’t agree. I take some solace from the fact that Google has tacitly agreed with this position by providing Django’s templating engine inside their App Engine.
One area that I think Django could do better is in error handling - especially in the way that exceptions get caught and rethrown. There are a number of ways that import errors or dangling named URL references can be misreported in stack traces or 500 pages; given that both of these are relatively common errors, it would be nice to provide better handling.
On a community level: I love the general disposition of the Django community. It’s polite and professional, and appreciates the value of good design.
If I had to pick a flaw, it’s probably the way that the core developers (myself included) sometimes let the community down in the way we communicate overall project direction. When plans are made (however informally), they aren’t always communicated effectively to the community. It also isn’t always clear (beyond the simple fact that we’re busy) why specific tickets with bug fixes don’t get committed to trunk in a timely fashion.
Even I get bitten by this sometimes - being based in Perth, Western Australia, the cost of travel prevents me from attending PyCon or the Sprints in person. As a result, I don’t get to participate in the in-person discussions - I sometimes have to read between the lines of blog posts and mailing list messages to work out what the plans are, sometimes confirming details using our secret core developer BatPhone. This is certainly an area where we could do better. Constructive suggestions are always welcome.

Shabda: Frameworks like Turbogears and Zope take a “best of breed” approach, and tie together a lot of existing components to build the full framework. In comparison, Django takes an integrated approach, and most of Django’s components are Django specific. For this reason, many people have accused Django of having a NIH. syndrome. What are the pros and cons of both these approaches? Why did Django rebuild most of its components?

Russell: Saying that Django rebuilt most of its components is one way of looking at it. The other way to look at it is to say that Django identified a core set of functions that need to be tightly integrated, and need to meet some specific design goals, and built tools that met those requirements. In some cases, it’s not even fair to say that components were rebuilt. Remember, Django went public in mid 2005, but existed as an in-house project at the Journal-World for some time before that. For some of these components, Django’s implementation predates the ‘existing component’.
Those that throw the NIH argument at Django are also ignoring two important points. Firstly, Django already uses a number of prebuilt components from elsewhere - the signal dispatcher, the JSON serializer, and any number of other components are taken from existing projects. We also have dependencies on external software - most notably in database backends, but also in serialization and markup engines.
Secondly, there is the presumption that plumbing together external projects requires no effort. This is patently false - you only have to look at the places that Django does rely on external projects to see why. The MySQLdb 1.2.2 prerelease bindings had all sorts of problems that require special handling; similar workarounds are required for unicode handling in recent versions of Markdown. Each version of these external components can potentially require different handling, sometimes in subtly incompatible ways. Once you introduce multiple interacting components, you introduce the problem of incompatibility between versions of those tools. Integration like this isn’t easy. The Django approach guarantees that everything works out of the box, and there is only one core configuration to document and test. It reduces the barrier to entry: making the first act of a new user a decision about which slightly different template engine will best meet their needs isn’t the best way to make a good impression. It also focuses the efforts of the community in testing, documentation, tutorials and advice. If improvements are required, the entire community is in a position to provide an opinion, not just the subset of the community that works with that component option. In a Balkanized world, there will always be a tendency for the question “How do I do X using tool Y” to be answered “don’t use tool Y, use Q instead”.
Building the components from scratch also guarantees that all the components fit - and I don’t just mean that they satisfy their mutual interfaces. There is also the issue that the core components should all ‘taste’ the same - that they follow similar design philosophies, apply similar assumptions, and so on. This sort of consistency is an invaluable resource, as it forms a kind of implicit documentation: once you have learned one part of the system, you automatically know how other parts of the system work. You don’t need to relearn the rules for each component.
One notable (and frequently made) argument against this approach is that you lose flexibility. While this is theoretically true, I don’t think it matters much in practice. For all the supposed inadequacies of the Django template engine, I challenge anyone to present a problem that can only be solved by using a different templating engine. On top of that - there’s absolutely nothing preventing you from using your own template engine, or your own database backend, or any other component if you don’t like Django’s offerings. If you don’t believe me, go look at Google App Engine.

Shabda: What are the focus areas Django testing framework? What utilities does this add too unittest or doctests? How does this fit in with browser based tools like Selenium?

Russell: Like most things in Django, the test framework aims to be as simple as possible, but no simpler. It provides an environment in which tests can be run, it provides the facilities for finding and invoking test cases, and provides a few helpful web-specific assertions that can be used during testing. Base Python unittests and doctests get a dummy database to work with, plus a dummy mail server; the Django TestCase extends unittest.TestCase to provide fixture loading and a dummy web client that can be used to programatically poke views. So far, the Django testing tools have concentrated on testing Django views as if they were ordinary Python methods - after all, that’s what they are. Selenium takes things another step, allowing for tests of how a web browser will behave when given the output of a view. I tend to think that testing the view itself is a more rigorous test, as it allows for direct inspection of view output. However, Selenium does have a place, especially when dealing with Javascript or other dynamic on-page interactions. Adding support for Selenium to Django is on the to-do list - it just requires someone to do the work.

Shabda: Thanks for the interview Russell.

This was the interview of Russell Keith-Magee, a longtime Django core contributor based in Perth, Australia. Next we interview Michael Trier, a longtime Django user and evangelist. Michael’s This Week in Django has been a great resource for the Django community, and we discuss Django, and how best to market Django. So stay tuned.

Oh and yes, 42topics.com is live now! (How are we different?) And we have a Django section. So regsiter let’s get rolling!

James Bennett is the release manager of Django, and a long time contributor. He works on Ellington, a CMS designed for news organizations. His book, Practical Django Projects, is being published by Apress, and is scheduled to hit bookshelves in June 2008. He graciously agreed to be interviewed at the 42topics.com blog. His blog, The B-List, can be found here.

Shabda: Would you tell something about yourself, how did you get started with Django, and what other OSS projects are you involved with?

James: I got into Django fairly soon after the initial public release; I’d been doing PHP and Perl work (mostly Textpattern on the PHP side, Scoop on the Perl side), and I was working on teaching myself Ruby and Rails because it looked interesting. But I’d always liked Python; it was just that there weren’t a whole lot of good options for Python web development at that point. You could do Zope, or you could do Twisted, but they both had pretty steep learning curves when compared to the type of work I was doing, so it just wasn’t worth it.
Django changed that; I think I did the tutorial the day it was released, and I just fell in love.
These days most of my open-source time is devoted to Django, and to various Django-based apps I’ve written and released. Back in the day I used to do the occasional bit of PHP hacking, and I wrote some plugins for blogging engines, but none of that stuff’s maintained anymore.

Shabda: What are your responsibilities as the release manager of Django? Who are the other core contributors of Django, and what are their current areas of focus?

James: My responsibility is largely bureaucracy. Within the project, I keep an eye on the various branches and their progress, try to stay on top of active/problem areas in the ticket tracker, and maintain a list of things that have to happen before 1.0.
Outside the project itself, I also stay in touch with people who are distributing Django — Linux distros which package it, etc. — and watch their Django-related bug reports. When there’s a security fix, I also send out an advance notice to them that we’re going to roll a release, and do the initial disclosure so they have some lead time to respond to that.
Outside of that, I also contribute to the documentation, as well as the occasional code patch, and I maintain the 0.91-bugfixes branch, which provides legacy support and security updates for some of the really ancient Django installs out there (our policy is to support the current release and two prior releases with security updates, which means 0.91, 0.95 and 0.96 right now).
The rest of the core team is just people everybody knows if they watch the developers’ list or the Trac timeline: Adrian and Jacob are the lead guys, obviously. Then there’s Malcolm, who you interviewed the other day, and who’s carved out a pretty big niche for himself: he did the bulk of the Unicode work and he’s doing queryset-refactor, he does a lot of maintenance on the i18n system, and he somehow still finds time to have a day job. I don’t know how he does that. Then there’s Russell Keith-Magee, (russelm in Trac), who’s contributed all sorts of useful stuff. He’s the go-to guy for testing and serialization, and just an all-around brilliant guy who fixes things left and right.
Gary Wilson and Joseph Kocherhans are also both big names you’ll recognize from commit messages; Joseph used to work at the next desk over at World Online, but now he’s up in Chicago with Adrian, working on EveryBlock. He’s done a lot of the grunt work on newforms and on laying groundwork for newforms-admin.
Ian Kelly and Matt Boersma keep our Oracle support working, and do an amazing job of helping to track down and solve obscure problems with that. And rounding out the core commit bits are Simon Willison and Luke Plant, who aren’t as active these days but can still be seen popping up every so often.
Out on the branches, we’ve had a lot of work done on newforms-admin lately by Brian Rosner, who’s been stepping up and helping to really get that sorted out, and there’s been a lot of unsung UI work by Christian Metts, who’s a designer at World Online but also a not-bad JavaScript programmer and can flaunt some Python when he feels like it. And then there’s the “gis” branch, GeoDjango, which is adding support for GIS — spatial querying — to the Django ORM.
Justin Bronn and Jeremy Dunck really kick-started that, Justin gave a nice demo of some of their work at PyCon this year.
And recently we’ve also been giving commit access to some of the translators, so they don’t have to go through as much bureaucracy to submit updated translation files; that’s sped some things up on the i18n front, because we’re blessed with a large number of people who are willing to pitch in and do that work.
Oh, and I can’t forget Wilson Miner; he doesn’t flaunt it that often, but he’s the guy who originally designed the Django admin, and he’s been known to make the occasional tweak to fix CSS stuff.
(hope I didn’t forget anybody there)

Shabda: Would GeoDjango be sometime merged with the trunk, or is it forever going to be a parallel branch to trunk?

James: The goal is that the GIS branch will merge, sometime after queryset-refactor lands, and it’s going to provide an application — django.contrib.gis — that you can use to enable spatial queries for your models. There’s a pretty good writeup on the wiki page of how that works, and they’ve been tracking queryset-refactor because it helps make some of the custom query construction easier.
I don’t know for certain if it’ll hit trunk before Django 1.0, but it will not be a branch forever; they’ve put in a ton of work on that, and I’m looking forward to getting to use it

Shabda: Once Django hits 1.0, would 0.91 be end of lifed, but support for .96 and .95 continue?

James: Like I said, the policy is we provide security fixes for the current release plus the previous two, so 0.91 would sort of fall off there after Django 1.0. But there are a lot of legacy installs out there that are perfectly happy for now, and I wouldn’t be surprised to see people unofficially continuing to submit patches and keep that maintained for as long as there are people who want to use 0.91.

Shabda: A little about you. You have majored in philosophy. I can think of one other, Paul Graham. Would you say, what you learnt from philosophy help with programming?

James: Well, I wouldn’t say there’s anything specific necessarily. But I think there’s a big place for people with liberal-arts backgrounds to come to programming, and I think philosophy’s a good path to do that. If you look at a typical philosophy program, you’re doing a lot of logic, a lot of critical analysis, a lot of abstract reasoning.
You have to get comfortable sooner or later with all sorts of formalisms that don’t necessarily have any practical meaning, and that’s very similar in a lot of ways to programming And when you get right down to it, as programmers, about 90% of what we’re paid to do is think: our job is to take a problem, analyze it, break it down into pieces and solve them.
And that’s not terribly different from what you spend four years doing in a philosophy program. I’ve actually joked about that a bit with some of my former professors, that I still get to argue as much as when I was doing philosophy, but the programming pays a lot better.
I do think, though, that there’s a big need for that sort of thing; we don’t really teach critical thinking anymore, and while it’s a vital skill to have no matter what you do for a living, it’s absolutely crucial to programming. So if you can get a good liberal-arts background where you’ve been taught how to look at things and pick them apart and analyze them, you can definitely do well as a programmer. Though it’d also be a good idea to take at least a few elective math courses…

Shabda: You have been working on Ellington, how is Ellington CMS different from, say, a customized version of Drupal?

James: Well, they have some things in common: both are modular CMS-style products, both are meant to be extensible.
But Ellington is really targeted from the ground up at news operations. There’s all sorts of specialized stuff in there that’s really optimized for the way a newsroom works, most of it culled from our experience as a newspaper, and of working with other papers.
So where with Drupal you’d really have to do a lot of customization because nobody’s really done this kind of “news all the way through” version of Drupal, with Ellington it’s there out of the box, and you hit the ground running.
I think we also have a very unique position because we are a news company, we’ve got a bunch of newspapers, some periodicals, TV news, etc.
And we work with those folks every day, we see how they do stuff, we hear about it when they run into problems, and so we’re in a good spot to see just what a newsroom staff really needs out of their online platform.

Shabda: Wordpress is PHP’s killer app, arguably Basecamp is ROR’s. What would you say Django’s killer app is?

James: Honestly I don’t know right now; I think the thing that Django wins on is that there doesn’t have to be the One Big App that everybody uses, instead there’s this huge blooming ecosystem of applications. In a way, it’s like asking what the “killer library” of the Python stdlib is; the killer feature is that you’ve got all that stuff available.
Though there are definitely some cool Django apps out there right now, and a lot more on the way. I think there’s a different mentality, though, because in general Python people seem to keep their heads down and just get stuff done. So it may be you don’t hear about some project until maybe they decide to do an open-space talk at PyCon or OSCON, and then you just get blown away.
There was a guy at PyCon who came up and showed me an app he’s been developing, and I won’t spoil it and give away what it is, but it made my jaw drop.
He’d taken something that’s a really common software niche that’s been dominated by these abominable products because it’s not really a sexy thing to be doing, and just absolutely nailed it. Guy’s probably gonna make millions. But I don’t know if there is a general-purpose “killer app” for Django right now, and I’m not really sure I want there to be one; people can see that sort of thing and think “oh, that’s all it does”. They look at, say, Ellington, and think “oh, this is only good for a newspaper-style CMS“, or they see Rails and Basecamp and think “oh, this is no good for me, I’m not doing a Web 2.0 thing”. So in a way I’m kind of glad we don’t have a “killer app” hanging over us and pigeonholing Django.
Though I should definitely give a shout out to Review Board; of the public Django apps I’ve seen, it’s probably the coolest, and again takes something that’s not usually sexy in terms of software and really nails it.

Shabda: As you said, “Python people seem to keep their heads down and just get stuff done”, do you think Django needs to a better job marketing itself. For example, I have pitched Django to a fair number of people, and I always have to start with with “Django what?”, as compared to say ROR, or PHP which seems to have a good brand recall.

James: Well, to some extent Django hasn’t done a whole lot of explicit marketing because we’re not at 1.0 yet, and I expect that after 1.0 it’ll both be a lot easier to do marketing and that there will be more of it going on. But at the same time, Django’s doing pretty well as it is; Google’s doing their App Engine stuff with Django bundled, there are startups using it to build the next big thing, and it’s even been quietly sneaking its way into some huge corporations. Plus it seems like every time I turn around there’s another book coming out.
I saw one yesterday at the bookstore downtown; I hadn’t heard about it until that moment, but I think that makes five or six books that’ll be out by the end of this year. So I think that’ll help. Digital Web magazine did a feature article on Django just recently, and I did an article for Sitepoint a while back as well. I like to view this as the phase where we build up momentum until eventually Django is an unstoppable juggernaut and everybody’s listening to gypsy jazz music ;).

Shabda: Everyone is pretty excited about your coming book. When can we have it in book stores? Would you give a brief overviews of what’s in it?

James: The book will, I think (and hope) be shipping around the end of June. It’s very much a hands-on introduction to Django: walking through building three applications, picking up progressively more advanced bits of Django as you go, and seeing some best practices in action. So you start out with just simple stuff, using the contrib apps and learning the basics of getting Django running. Then you do some simple customizations of admin templates, then start building some models and views, then on into building full-on applications.
There’s not room to cover every single thing you can do with Django, but I think it handles a pretty good spectrum of techniques from basics up to some advanced things that let you poke around and really get a feel for how stuff works.
And of course, you get periodic bouts of me up on my soapbox yelling at people about how to write reusable applications, because that’s what I do.

Shabda: You talk a little about App Engine in Batteries sold separately. What effect do you see of App Engine on python hosting ecosystem, on Python web applications, and on Django?

James: I honestly don’t know what to expect from App Engine. It’s a very different kind of thing from what anybody outside Google is used to, and it’ll probably be a while before it’s really shaken out and we get an idea of the impact it’ll have. I fully expect that Google’s going to start supporting other languages, so there won’t be this effect where you’ll always have to use Python to use App Engine. And the way they’ve sandboxed it is going to make it feel weird to Python people, I think. And that’s on top of getting used to the fact that you’re not using an RDBMS; I’ve been watching the blog flamewars about that, and that seems to be the big takeaway.
If I had to make a prediction now, I’d say that App Engine will bring some people to Python, but probably not in hordes, and that its big long-term effect is going to be to point out to a lot of folks that they’re not really using the “R” in “RDBMS“, and so maybe it’s OK to think about their applications in a different way. And that’s not Python-specific at all.

Shabda: What were the focus areas of the newforms-admin branch. How far have they been achieved. About when would the newforms-admin branch be merged? How backwards incompatible is this going to be?

James: Well, there are a couple goals running parallel. The first thing, obviously, is that with the oldforms package deprecated we’ve got to get stuff migrated to newforms, and the admin’s got to make that jump.
And because the admin does a lot of tricky and complex stuff with forms, it’s also been a fertile proving ground for designing some advanced newforms features that’ll find their way back into trunk and make it easier to do that tricky stuff on your own.
Another big thing is cleaning out the coupling issue where you declare a class inside a model to activate the admin interface; there’s no reason for that to happen, so instead there’s a class you’ll subclass and override stuff on to customize the behavior, and then you’ll register a model to have the admin interface you’ve set up that way.
Finally, there’s been a lot of stuff abstracted in newforms-admin; while I don’t think it’s really an official design goal, there’s been a lot of opportunity to provide hooks where people can go in and customize stuff. I looked at the code a couple months ago, and it was about 95% of the way to being able to run completely without django.contrib.auth, for example, because there are enough hooks where you can override something and replace defaults to make it use something else.
And that’s a huge deal, because it means you could do stuff like run the admin on HTTP auth, or run it in a corporate environment against an LDAP database, and you barely have to touch anything; you can already do an auth backend that handles most of the work, then you tweak a few things for the admin and you’re good to go with no hacking directly on Django code.
Plus, a lot of people will be happy to see that the methods you can go in and override al