Thoughts on Web Development
I recently attended the CodeMash conference in Sandusky, Ohio for the first time this week. This was after hearing that I should go for the past three years from friends and former colleagues in both the .NET and Ruby communities.
CodeMash started as a mostly a regional conference, but there were quite a few of us in town from outside of the Ohio/Great Lakes region. Microsoft also chose to officially launch its new WebMatrix platform at CodeMash so the conference got international attention this year.
I’ve attended a few conferences, but the energy at this conference was just infectious. This was the first conference where I was up till at least 2 AM every night due to the level of engaging discussions going on.
I came away with lots of important insights about the future of the .NET platform, the power of Unix based platforms for cloud computing, and proven strategies to break into entrepreneurship.
As for formal sessions, I attended a mix of .NET, Ruby, and Cloud Services sessions. While some of the sessions were interesting, as in most conferences the really engaging discussions happened at the hotel bars and in the hallways.
The connections that I made after attending this conference will be invaluable. I was blessed to be able to finally meet thought leaders like Chad Fowler and personally thank them for their contributions to the community and my own career. I was also fortunate to meet many folks that I’ve conversed with on Twitter over the past few years. And as always, it was great to reconnect with my former Telligent colleagues from around the country as well as my friends from the .NET community in Austin, Kansas, and Iowa.
I was very impressed with the organization of the CodeMash committee and their professionalism in putting together a great conference. I’m hoping to make it to CodeMash every year from here on out. If you’re a .NET, Java, or Open Source developer, you should do the same.
I was very fortunate to recently attend the Pablo’s Fiesta Open Spaces conference in Austin, TX. This was my first Open Spaces event and I thoroughly enjoyed the experience. It was also great to catch up with some folks in the community that I hadn’t seen in a while. And I enjoyed meeting many folks with whom I had only interacted with via Twitter, email, or blog comments.
I attended many sessions during the conference, and I loved that hopping around from session to session was encouraged to stimulate hallway conversations and keep things interesting. I also found the Pablos Fiesta wiki very useful since I could read any notes on sessions that I missed.
The sheer intelligence of the folks at this conference was mind blowing. I had expected the crowd to be mostly Austinites, but I met some really sharp developers from Kansas, Iowa, and Oklahoma that all really impressed me. I felt inspired by the passion and intelligence exhibited by these developers that are coming out of smaller cities and working in small companies. That passion is something that I feel is lacking in many of the developers I meet in the Dallas-Ft.Worth area. I think the potential is there though, and this conference inspired me to help bring some of that passion to NorthDallas.NET.
One thing that came up in some discussions was the growing gap between Mainstream .NET and ALT.NET. Some folks accepted this gap as inevitable since the .NET developer community is so large and such gaps are inevitable. I disagree with this stance, as I think the gap makes it difficult to find good developers to work with, good environments to work in, and ultimately has turned a few people away from the platform all together.
At my current job, I’m reminded nearly every day of the true cost of that gap and find myself trying to bridge it quite often in order to make progress on our larger engineering initiatives. While it’s certainly easier to take a hard stance and accept this gap, I’ve seen first hand the effect that bridging the gap can have on the right individuals. And how great ideas can spread like wildfire through the organization once they have the right champion.
I want to express my gratitude to the members of the Los Techies blogging community, many of whom clearly worked very hard to make Pablo’s Fiesta possible. I hope that this event will become a regular thing and that I get another chance soon to learn more from this community.
I mentioned in my previous post that I believed much of the backlash against TDD is more technical in nature than many in the .NET developer community have realized. This post is to expand on that thought a bit further.
Many of the issues that .NET Developers have around doing TDD in C# is the requirement of using a large number of Interfaces around dependencies to make classes more testable. Writing your code this way requires practice, discipline, and most importantly, the appropriate tools.
I’ve long since reached the conclusion that the out of the box developer tools provided by Microsoft are wholly inadequate to using a Test First methodology. MSTest has always been a weaker alternative than NUnit, despite having nearly 5 years to catch up. Visual Studio still does not provide enough refactoring support, code navigation, or targeted code generation without an add-in like Resharper or CodeRush. Truly understanding TDD requires a mastery of these tools and others like Rhino Mocks and StructureMap.
Recently I’ve taken it upon myself to relearn Ruby on Rails after a long hiatus. Juxtaposing the Ruby ecosystem against the .NET ecosystem has been eye opening. For one thing, as a dynamic language, many of the technical barriers of doing TDD don’t exist in Ruby. Mocking and Dependency Injection have little to no value in Ruby as neither one is required to unit test your code. On the flip side, common compile time errors in a static language like C# can become runtime errors in a dynamic language like Ruby. The frequency and potential for these kinds of errors makes unit testing no longer optional when coding in Ruby.
Also, the Ruby community uses surprisingly few developer tools. Most Ruby developers use the most basic of text editors like VIM and TextMate to write their code. They do use a lot of libraries (or gems as they call them), but most of these are free and open source. Not having a corporate leader like Sun or Microsoft has forced the community to be far more self sufficient and create their own ecosystem. It is also evident that all of the thought leaders have also made significant open source contributions.
The .NET community has made strides in the same direction, as many of the new leaders at Microsoft like Phil Haack and Scott Hanselman came from the developer community and have made significant open source contributions. However, I believe the core technical issues that surround a statically typed language will only continue to hurt the platform’s growth. Ruby and Python will only continue to make gains in performance and productivity. Eventually Microsoft will have to respond with a suitable dynamic language. This may be coming sooner than expected based on John Lam’s departure from IronRuby to form a new team that is working on a new secret project.
It seems as though the .NET developer community is becoming more polarized regarding their thoughts on the value of Unit Testing. I see it as a split between those that are dogmatic about Unit Testing, and those that are in denial that it has any long term value.
Some of this was spurred on by Cashto’s “It’s OK Not to Write Unit Tests” post, but I think that sentiment has been brewing for quite some time and it just hadn’t properly been put into words.
I think Cashto had some valid points and that in certain instances, Unit testing and TDD can become cumbersome. In my opinion, the following would need to be true to get away with not Unit Testing.
While some may consider these three conditions to be unrealistic, there are certain small lean startups or small consulting projects that can operate under these assumptions.
The problem with the .NET community’s reaction to cashto’s post was that the dogmatic individuals immediately dismissed it and the ones in denial unconditionally embraced it. The dogmatic strictly believe in a test first methodology, and are quick to dismiss any notion of the contrary. The irony is that their dogmatic attitude is reactionary to the “no testing” attitude previously espoused by many in the community, especially those most closely associated with Microsoft. Those individuals who fully embraced cashto’s post and cried “Unit Testing is Dead” are the ones most often in denial that never really gave it a chance. These individuals may have dipped their toes into Unit Testing years ago and quickly gave up without putting enough initial effort or seeing any results.
For those of us who fall in the middle, the path has become unclear. The dogmatic approach can be off putting, especially when faced with the often harsh attitudes of those espousing it. Staying in denial, however, is not an option for those who have been in the trenches and seen more than a few nasty bugs caught with good unit tests.
I have chosen to look outside the .NET community and see how the Rails community has dealt with this issue. I suspect that many of the disagreements are actually more technical in nature than most people realize. More on this in a future post.
Software can often be a challenging product to sell. If you don’t have a great sales team, it can be difficult for anyone to know just how good your software is.
Many great companies with fantastic software have fallen flat due to a lack of sales revenue that resulted in them being eclipsed by a poorer quality competitor. This phenomenon has prompted many software companies to fore go quality in return for beating their competitors to market. The idea being that even if your product is of poor quality, your customer base may be locked in long enough to give you time to actually meet their needs. Twitter is a perfect example of this.
While I think there is some validity to this thinking, too much of it can lead to a very bad pattern that I’ve seen in nearly every software company that I’ve worked in. It goes something like this:
The main problem here is that there is zero accountability for the Sales team’s actions. Usually, Sales is a separate and independent department that is measured on their pipeline, not on long term relationships with customers. Upper management is often reluctant to restructure organizations, as it might result in lost sales. Also, because of the way sales commissions are distributed, a net operating loss from a customer never affects Sales teams’ compensation.
As a developer on the other side of the fence, it is easy to point the blame at the Sales teams for their irresponsible behavior. However, it should be upper management’s responsibility to build in accountability for customer relationships with sales teams so that this type of situation does not repeat itself. While Sales are a very important part of any business, left unchecked they can wreak havoc on the long term viability and reputation of your organization.
As I mentioned before, my new role in the North Dallas .NET User group is to help organize speakers for our user group meetings. I’m excited to share the upcoming speaker lineup as I think we have some great speakers on some very interesting topics.
November 04, 2009: Rachel Appel on JQuery
January 6, 2010: Caleb Jenkins on Model-View-ViewModel in Silverlight
February 3, 2010: Chris Patterson on Event Driven Architectures using Mass Transit
March 3, 2010: Joseph Hill on MonoTouch
April 7, 2010: Matt Hinze on Practical Inversion of Control
Our meetings are always on the first Wednesday of the month at the Intuit office in Plano, TX. More info on our group is available at NorthDallas.NET.
I recently came across an old post on The Mole Skin that listed out the 5 deadly sins of the Web Profesional and I thought it would be appropriate to come up with a similar list for Software Developers. So here is my list.
Being part of a developer community is an essential aspect of your personal growth as a developer. As developers, many of us are not very inclined to network. Also we often think that the necessary skills needed to do our jobs can be self-taught. While you might be able to learn the basics of a technology on your own, you might not learn the most efficient way to use it. Participating in a community will help you to meet like minded people that you can relate to and help further both your technical skills and your communication skills. These can lead to better jobs, better solutions, and new challenges.
While communication skills are key to being a good contributor, sharp technical skills are what separates the average developer from the truly great ones. Unfortunately, it is very easy to let your skills stagnate. Maybe you’re in a position that started out with a very cutting edge codebase, but after 3-4 years most of the technology is considered legacy. Or perhaps you are already in a maintenance-only role working on legacy technology. In either situation, your market value as a developer will continue to deteriorate over time. While changing jobs might be an option, there are other things you can do to keep your skills sharp. One option is to contribute to open source projects. Another option is moonlight with paid side projects. Or you can start building a product/website in your spare time on a technology stack of your choosing. Even if your after hours endeavors are not as successful, you’ll still be learning and staying sharp until the time is right for you to change jobs.
A lot of times you may start a position with a lot of enthusiasm and a desire to really make improvements to the product you are working on. But after a while and for any number of reasons, you may feel beat down, burned out, and generally uninterested in the direction things are going. Maybe you used to work lots of extra hours without feeling overworked, but now even 40 hours of work is painful. Despite all this, you don’t do anything to change your situation. You’ve officially transitioned from having a career to just having a job. This is one of the worst things you can do as a developer, because it will lead to both a stagnation of your technical skills, and slowly kill off your passion. If you see yourself veering down this path, make the necessary changes before it’s too late.
A lot of times when developers pick up new technologies, they do a lot of experimentation and change layers out often. This constant changing of your technology stack is good for learning, but really bad for shipping. Most seasoned developers have learned that there is no silver bullet. Every new framework that claims to change the world will have some limitations that you won’t find until you’re knee deep in an implementation. When you get stuck, there’s always the temptation to switch frameworks. Or maybe you started on v1.0 of a framework and when you’re halfway through, v2.0 comes out so you’re tempted to upgrade. Such switches are often very costly and can cost you time, money, and even damage your reputation by labeling you as indecisive and unable to get the job done. Resist the temptation to always go for the new and shiny by limiting the unknowns in your technology stack. It’s always good to try new things, but not at the cost of getting your software to ship.
Letting your skills stagnate and “staying and quitting” are signs that you’re killing off what ever passion that you had when you first started your job. When your passion dies, so does your creativity. It will become obvious to those you work with that you’ve reached your peak. Those who looked to you for technical guidance may look elsewhere for input. You’ll continue to only get uninteresting work that other more passionate developers have passed over and the situation will only get worse.
Web Developers that lack skills in User Experience Design often turn to Designers to flesh out websites via the creation of Comprensive Layouts (or comps). Comps are usually created in an image editor like Adobe Photoshop or Fireworks and are exported as static images, usually in the form of Photoshop (PSD) files. These images are then “sliced” into HTML/CSS markup and incorporated with both server-side code and client-side code by the web developer.
For those developers coming from a more traditional Application Development background, the inclusion of Designers adds a new element to an often neglected part of a developer’s workflow, the User Interface. While this can present challenges for inexperienced developers, it is the preferred way to launch high profile sites on the web today and the User Experience can be far richer.
I first gained exposure to this type of environment when I was a developer on the Professional Service team at Telligent. The ideal project flow for sites that we launched on Community Server went as follows:
Unfortunately, as noted by Jon Galloway, this perfect workflow rarely exists in the real world. In my experience, this workflow is extremely inflexible to minor changes and hiccups. Here are some of the common problems I ran into while doing this kind of work at Telligent:
Because of the strong segmentation between Designer and Developer, there’s not much flexibility when direction changes and web development firms often build in large amounts of buffer time and high margins to make up for this.
I think web development firms that have put most of their energy behind ASP.NET are at a distinct disadvantage here. In general, most ASP.NET applications are line of business applications with limited complexity in the UI. Thus, most developers on the .NET platform have limited experience with cross browser quirks and don’t have enough CSS/HTML expertise. In addition, only recently with ASP.NET MVC has the platform provided sufficient control of the generated markup. For a long time, ASP.NET developers had to write their own Custom Controls or work within the constraints of the built-in primitive controls. Such custom controls often provided the necessary markup control, but usually required a costly maintenance cycle with difficulty making changes later on.
Other platforms such as PHP and Ruby On Rails were built with reduced friction between the Developer and the Designer. Most PHP and Rails developers can wear both hats when needed. I’ve worked with such individuals and I find that their workflow is far more flexible. Many of these developers are able to skip the comp step completely and work directly in markup. What they lose in speed by staying outside of Photoshop, they gain back in having flexible designs that can respond to change quickly. And because their chosen development platforms are much friendlier to markup, the move to Service Side code is much more seamless. The downside is that their resulting code may have a lot of mixed UI/Data concerns, may not be very maintainable over the long run, or may not perform well under large amounts of traffic. But due to the limited shelf life of most of the sites, these issues can have a limited impact on the developer’s overall success.
I think it will be interesting to see how tools like SketchFlow and Expression Blend help blur the lines between developer and designer. And I think the potential is there to create a better workflow. Coming from an environment where UX was a first class citizen, I certainly miss the inclusion of Designers in the process. Hopefully one day some of the ideas coming out of the boutique shops will work its way into mainstream software development.
There’s an interesting discussion going on right now about whether Software Development has grown too complex. Lately this discussion was sparked by Ted Neward’s post on Agile treating the symptoms, not the disease. I’ve been interested in this discussion since I first heard the DevLink panel discussion on the same subject. My general feeling is that while it’s unfortunate that software development has become too complex for amateurs, a higher barrier to entry is probably better for the industries that require software.
Phil Haack’s post is good because he talks about how he made a living cleaning up other people’s messes. I’ve had a very similar experience in my career both when I was consulting and as a product developer. For every successful business that started on Access, there’s likely a more successful business that cleaned up the mess left behind by Access. But while developers can make money on both sides, the businesses using these applications are paying a high cost to get the software written once and rewritten later.
I once worked on an application that started out as an Access database. It later migrated to local instances of MSDE/SQL Express, then finally to SQL Server in a hosted environment. Each of these migrations were performed soon after the application and its team were shuttled along from one corporate acquisition to another. Along the way, a large amount of underlying technical debt was never addressed and soon the codebase became a big unwieldy mess. Many customer requests were ignored or deemed too costly and the application stagnated until the last acquiring company killed it off, leaving its existing customers to look elsewhere to solve their business needs.
It’s true, the product had made the original company that built it a lot of money. But the customers had paid a substantial amount into its ongoing development with little to no gain in most cases. Sounds like a raw deal for the customers, doesn’t it?
Unfortunately, the customers in this case were hospitals. And as their costs went up, those costs were passed down to patients in the form of higher insurance premiums and a higher cost of care. This is just one example of the trickle down effect of wasted money on software and how we as consumers end up paying for it in the end. I’m sure there are similar examples in the Defense, Airline, and Hospitality industries too. While there are some industries such as Social Media where a low barrier to entry has spurred a lot of good competition, I don’t think this is good for every industry out there.
For this reason, I believe software should be harder to write and there should be a barrier to entry. We can’t continue to let such businesses waste dollars in our economies on poor software implementations that constantly need fixing. Keeping the barrier to entry high forces businesses to get it right the first time. In the process we will have to sacrifice some agility and time to market, but I believe the businesses in our economies will be better off in the long run. They will then be able to direct their investments towards more worthy pursuits instead of wasting money fixing or replacing bad software that should have worked right in the first place.
There’s a lot of discussion going around about the evolution of software development towards building in long term quality and maintainability up front using practices like TDD. A lot of it was sparked by Joel Spolsky’s praise of the Duct Tape Programmer. The latest post in this discussion that caught my eye was Scott C Reynold’s Quit Living in The Past – Practices Evolve.
While I agree that “time to market” is an important driver for software, it shouldn’t be the catch-all excuse for building low quality solutions. Too often I’ve seen time to market gains get erased by the high cost of software maintenance and production support down the road. The current codebase that I’m working in on a daily basis was written by a few Duct Tape programmers and now my company is paying a high cost to satisfy existing customers while attempting to increase market share with a creaky solution.
The fact is, many businesses keep code around for a lot longer than we as developers would like. Software has become the life blood of many businesses and is very costly to construct, so getting applications out the door quickly and planning for a full rewrite every few years is rarely a good business decision.
Good developers are also becoming scarce, especially when looking for those that have recent experience with legacy technologies. This is because developers with talent are wanting to constantly evolve into new technologies and stay challenged. The only way to retain this talent and utilize it is to have a constantly evolving codebase. If the foundation of the software incorporates a lot of technical debt that is completely dependent on legacy technologies, a business will have a lot of trouble finding and retaining skilled developers to work on it.
These conditions all appear to run contrary to each other. Businesses want to keep working code around, but Developers don’t want to keep working on the same code. Development Managers have to keep their businesses happy, but still find and retain skilled talent.
So what’s the answer? Build quality and long term maintainability into the codebase up front. Abstract your Persistence from your Service Layer so that you can swap out data access technologies every few years. Separate your Domain from your Presentation so that you can swap between web technologies when appropriate. Make unit tests a requirement for your codebase so that you always have a solid safety net when you break ground on new technologies. Build a suite of automated acceptance tests against your User Interface so that you’ll know that something broke long before your customers find it. Release early and release often so that your software doesn’t stagnate and you can maintain and increase market share.
The Web Gambit 