Brokenwire.NET :: Stuff to read

Not a K2 insider anymore…

2011-09-01T10:21:32Z

Don’t worry, I’ve just moved a bit closed to [the] Source[Code Ltd]. I’ve worked at Capgemini and its predecessors for 5 years and it was time to do something different. Capgemini is a wonderful company with lots of opportunities. I’ve also learned a lot there and I’ll always look back at it with good feelings.

I’m now working for K2NE GmbH. K2NE is a K2 distributor run by Ruan Scott who, if you have been in contact with K2 in Europe, you’ll probably know. K2NE operates in the North of Europe, covering Scandinavia, Germany, Swiss and Austria. The opportunity I have at K2NE is a large challenge for me that I could not resist!

Unfortunately, K2 distributors are not allowed to be Insiders so I’m losing that hubris (always wanted to use that word on my blog). That doesn’t mean this blog is going to stop or won’t be updated. I’m still keeping it and I hope to bring more details/real life problem fixes on this blog, together with the new things! And I can assure you, there’s so much goodness to come!

I also want to thank K2’s insider program because it helped me keep up my knowledge and start work at K2NE so easily!

startup

2011-07-19T15:30:36Z

Little Geeks

2011-08-09T21:30:56Z

Password Strength

2011-08-10T04:00:00Z

What curious property does this string have?

2011-07-12T19:19:00Z

There are all kinds of interesting things in the Unicode standard. For example, the block of characters from U+A000 to U+A48F is for representing syllables in the "Yi script". Apparently it is a Chinese language writing system developed during the Tang Dynasty.

A string drawn from this block has an unusual property; the string consists of just two characters, both the same: a repetition of character U+A0A2:

string s = "ꂢꂢ";

Or, if your browser can't hack the Yi script, that's the equivalent of the C# program fragment:

string s = "\uA0A2\uA0A2";

What curious property does this string have?

I'll leave some hints in the comments, and post the answer next time..

UPDATE: A couple people have figured it out, so don't read the comments too far if you don't want to be spoiled. I'll post a follow-up article on Friday.

VS 2010: How to find out if VS SP1 is installed?

2011-07-11T09:08:00Z

Question: How can I easily see if VS SP1 is installed on my machine?

Answer: Run VS 2010, open Help » About. Look for "SP1Rel" after the version number.

Screenshot:

large slide deck: technical meeting automated testing with vs2010

2011-07-10T19:25:03Z

Slide deck from a 3 hour presentation (in the north of the Netherlands) covering automated testing with vs2010: test case specification, execution, automation and test infrastructure with VS2010 CodedUI and Lab management.

Technical meeting automated testing with vs2010

View more presentations from Clemens Reijnen.

Verdacht pakje legt treinverkeer Madurodam uren stil

2011-07-02T11:27:34Z

Een onbeheerd pakket heeft woensdagochtend in Madurodam voor opschudding gezorgd. Vanwege de grootte van het verdachte object werd vrijwel onmiddellijk al het treinverkeer in de omgeving stilgelegd. Ook werd het gebied om het pakketje heen afgesloten. ‘Het verdachte pakketje blijkt voor zover wij nu kunnen zien te bestaan uit een kubus van grote brokken samengeklonterde suiker, ingepakt in papier,’ aldus de politiewoordvoerder van Madurodam. ‘Wat de herkomst van het pakket is is nog onduidelijk, onze experts zijn er nog wel even zoet mee.’

De burgemeester liet weten dat er vooralsnog niets aan de hand is. ‘Laten we het voorval niet groter maken dan het is,’ zegt hij in een verklaring. Wel rukte na de vondst de brandweer uit, maar die werd meteen daarna weer teruggeroepen. Enkele minuten later rukte de brandweer wederom uit, om vervolgens opnieuw teruggeroepen te worden. Niet lang daarna rukte de brandweer opnieuw uit.

Het vliegverkeer op het vliegveld had geen last van de vondst. Alle vluchten konden hun gebruikelijke rondjes blijven taxiën. Voor de gestrande treinreizigers werden bussen neergezet. Het incident roept herinneringen op aan de inslag van de komeet Rucanor, vorig jaar zomer. Toen sloeg een ronde astroïde opgebouwd uit zeshoekige leren vlakken in vlakbij het Rijksmuseum en werden enkele stilstaande auto’s verwoest. Of de vondst van het pakketje enig verband houdt met dit incident is nog onbekend.

‘We zijn behoorlijk geschrokken,’ aldus een omstander. ‘De meesten van ons zijn aan de grond genageld van schrik.’De reizigers op de stations van Madurodam verklaarden weinig hinder te ondervinden van de stremmingen. ‘In principe is alles hier op loopafstand’, aldus een forens, ‘plus, de meesten van ons gaan toch nergens naartoe.’

Arjen Lubach is gastschrijver. Onlangs publiceerde hij Magnus bij uitgeverij Podium. Voor meer informatie, zie http://www.arjenlubach.nl/

Writing naked (nakeder than Orwell)

2011-06-27T09:50:00Z

Here are Orwell's rules, edited:

1. Never use a metaphor, simile, or other figure of speech which you are used to seeing in print. You don't need cliches.

2. Never use a long word where a short one will do. Avoid long words.

3. If it is possible to cut a word out, always cut it out.

4. Never use the passive where you can use the active. Write in the now.

5. Never use a foreign phrase, a scientific word or a jargon word if you can think of an everyday English equivalent. When in doubt, say it clearly.

6. Break any of these rules sooner than say anything outright barbarous. Better to be interesting than to follow these rules.

The reason business writing is horrible is that people are afraid.

Afraid to say what they mean, because they might be criticized for it.

Afraid to be misunderstood, to be accused of saying what they didn't mean, because they might be criticized for it.

Orwell was on the right track. Just say it. Say it clearly. Say it now. Say it without fear of being criticized and say it without being boring.

If the goal is no feedback, then say nothing. Don't write the memo.

If the goal is to communicate, then say what you mean.

My best tip is this: buy a cheap digital recorder. Say what you want to say, as if the person you seek to persuade is standing there, listening. Then type that up. Simplify. Send.

The Debugger Mindset

2011-06-17T22:41:12Z

Today I want to talk to you about a serious mental disorder that affects far too many programmers.

The symptoms of this disease are many:

Frantically saying “It’ll work this time”
Randomly adding +1 and –1 to integers
Inability to see the forest
Irritability
Irrational paradoxical statements like “I am sure this is impossible, but I don’t know how it is happening.”
Screaming
Spitting at hamsters
The list goes on…

What I am talking about is the debugger mind set and…

It is a mental disorder

So what is this debugger mindset I am talking about?

It is more than just overusing the debugger, it is thinking and acting in a way that is the same way one tends to think and act when they are stepping through code in the debugger.

I am guilty of it as I imagine many of you are.

And it is a disease.

A debugging tale all too familiar

Picture this… You are just about to check in the final piece of code that you have been working on all week for this awesome feature you are developing and something is wrong.

It is just not working. You can’t figure out why. The behavior of the code seems to defy all logic.

You set a few break points and start stepping through code. (By the way it is 4:00 on a Friday and you were about to check in this code and go home a little early, you deserve it.)

Step… step… step… Check this variable… Check that variable… Evaluate this expression.

Nothing, no clue as to what is going on. You are beginning to sweat. The release is going out on Sunday. You need to get this fixed or the code can’t release.

You look at the clock. It is now 5:03. You glance at your phone and see you have a text from your wife. No time to look at what it says. You are intensely focused. You must solve this bug.

Step… step… step… Check this variable… Check that variable… Evaluate this expression. You begin to sweat more.

You can’t understand why this one little thing isn’t working. It is driving you nuts, it doesn’t make any sense… You start scratching at your head in frustration.

Time ticks by, it’s now 6:30. Your phone rings, it’s your wife.

“Did you get my text? When are you coming home? It’s Friday. It’s the weekend.”

You fumble to get to your SMS program on your phone. You don’t have time for this, you must SOLVE this bug, now! Grrr, you are getting angry and irritated. You try and hide it, but it comes through in your voice.

You’re thirsty, you’re hungry, your eyes hurt from not blinking. You’re tapping your foot like a madman intently focused on the keyboard. It’s now 7:40.

You start frantically changing variables. You’ve lost track of what the code is doing now, instead you’re just watching values in your watch window. It’s getting hard to think, minutes turn to hours.

You finally solve your bug at 11:03 PM, but your nerves are shot. The taste of victory is soured like milk left too long in the fridge. You can taste the chunky curds in your mouth as you think about how even though you fixed the problem, you still don’t understand why your solution worked or what was causing it.

It is very important to avoid this pattern of behavior

And it can destroy your productivity like nothing else.

You don’t actually have to be in the debugger to have this mindset, even though it often occurs when you are.

All developers are susceptible to it. I know I have spent my share of time thrashing around with it. It takes a fairly simple 1-hour-to-solve-problem and makes it take 8 hours to solve because we try to solve it in 5 minutes.

Very few things in life have a 5 minute solution.

When we start thinking this way, we lose all ability to think rationally and actually solve a problem. Instead we start zooming into symptoms and using frustration instead of wit.

This kind of mindset can put us into a downward spiral of burnout and insatiability. We end up wasting time and losing focus. It is the most frequent time in my career when I have made stupid mistakes.

How to avoid the debugger mindset

The is a perfect storm of conditions which tend to cause this kind of a mindset, mix and match these as you will:

A looming deadline (self-imposed or otherwise)
Large chunks of almost complete work that can’t stand on its own
Being in a hurry and looking for a shortcut
Believing a problem is probably simple
Believing a problem is probably hard
Jumping to the debugger too soon
Being at the end of a marathon session
Not taking enough breaks

There are many other conditions of course which can contribute to this mindset, but these are some of the ones I find most common. It can only take a few of these in combination to really push you over the edge into no man’s land.

Avoiding the debugger mindset can be as simple as trying to eliminate as many of these precursors as possible.

Here are a few tips I have found to directly counter this list above:

Don’t impose deadlines on yourself and try to ignore unrealistic ones that are imposed on you. Instead try to focus on doing a good job and being productive. You are much more likely to get fired for being in the debugger mindset and making a stupid brainless mistake than for taking a bit longer to accomplish a task or having to tell your boss that you cannot in good conscious commit to a deadline. Never forget that software development is always going to have a huge “unknowable” factor to it. No one can blame you for that and it is not your fault.
Try to commit as frequently as possible and divide your work into very small tasks. Smaller is better. Also try to split work vertically rather than horizontally. This means trying to implement a feature in terms of small working pieces that work end to end, rather than trying to implement all the functionality in a single layer at a time.
There are no shorcuts in programming. When you need time take it. Don’t forget to take time to quietly think about a problem. Quiet contemplation is work. It is not slacking off. Take a walk if you need to. Ask for a second set of eyes before you get into panic mode.
Do not make an assumption about a problem until you understand fully what the problem is. Don’t tell yourself it is easy, don’t tell yourself it is difficult, instead refrain from making a judgment call until you have enough information to do so.
Don’t prematurely go to the debugger. Always try to solve problems without the debugger first. Once you have gone to the debugger, you are much more likely to get sucked into the debugger mindset. If you do need to use the debugger, always have a hypothesis of what the problem is and use the debugger to confirm or deny it.
Do not use the debugger to help you understand what the code is doing. Instead use unit tests for that or refactor to make the code clearer. Divide and conquer the problem into a series of branching hypothesizes. Use the debugger only when necessary to confirm or disprove a hypotheses and then form a new one and repeat all the while remaining calm and understanding that you don’t know how deep a rabbit hole is until you go down it. (This StackOverflow question points out a fairly extreme view of what I am saying here, but I tend to agree with it more than I disagree with it. I won’t say never use the debugger, but I will say that I myself use it too much and jump to it too quickly.)
Take breaks and don’t work on the same piece of code for too long a period of time. As stated above, break your tasks into smaller tasks so that you are always near a checkpoint. If you are feeling frustrated, take a 5 minute break to clear your head and regain your focus.

What do you think? Have you ever been in the debugger mindset? What are some ways you avoid it?

As always, you can subscribe to this RSS feed to follow my posts on Making the Complex Simple. Feel free to check out ElegantCode.com where I post about the topic of writing elegant code about once a week. Also, you can follow me on twitter here.

A15 wordt als tolweg doorgetrokken naar A12

2011-06-16T07:35:55Z

We krijgen er in Nederland een vierde tolweg bij. De A15 eindigt momenteel bij Bemmel tussen Arnhem en Nijmegen, maar zal worden doorgetrokken naar de A12 en aansluiten tussen Duiven en Zevenaar. (more…)

Application that examines the assertion that...

2011-06-07T18:32:07Z

Application that examines the assertion that continuously following the first link of any Wikipedia page eventually leads to 'Philosophy'.

(Want more? See NOTCOT.org and NOTCOT.com)

Introducing Octopus, my MicroISV

2011-06-05T04:00:00Z

A few weeks ago I posted some screenshots of an application I'm working on. A few people guessed, rightly, that it is a tool for automated deployment. Progress is going well, so I'd like to share some more details, and get your feedback.

Octopus is a automated deployment solution for .NET applications, powered by NuGet and designed for convention over configuration.

Deploy Web applications and Windows Services
Configure your applications automatically
Works inside your network or over the internet, securely
Track and report on how releases are promoted between environments

I'm working my way through the SecretGeek 25 things list, and writing plenty of code. My goal is to have Octopus ready for beta testing in a few weeks. If you'd be interested in taking Octopus for a spin, add your email address below. I promise to only send occasional updates as the beta program opens.

Enter your email address:

Where Octopus fits in

Imagine your development team has been charged with building an intranet web solution. The solution consists of:

An ASP.NET MVC web site
A SQL Server database
A Windows Service for performing some long-running tasks

The team are working iteratively. They check code into source control, and the build server picks up the code, compiles it, and produces artefacts ready to be deployed. Now, someone has to figure out how to deploy it.

Every application has different deployment needs, but they are usually pretty similar:

The CI server builds the code and products artefacts (Zip files, WebDeploy packages, MSI's, etc.)
The artifacts are copied and extracted on the target server(s)
Web/App.config settings and connection strings are updated
Performance counters and event log entries are created
IIS web sites are configured
Windows Services are installed and started

Ideally, these deployment steps would be automated. Unfortunately automating these deployments still takes a lot of manual work. Usually the solution is a cobbled together band-aid solution of PowerShell scripts, XPath queries, batch files and point-and-click tools.

Once you've cobbled together the PowerShell scripts and MSI packages for your application, zoom out, and look at the rest of your organization. How many other projects do you have? How many of them have automated deployment? How different is the deployment process for each application? Does it need to be this hard?

Octopus in action

Here's where Octopus fits in a typical organization:

Developers check code into source control
The CI server (TeamCity, TFS, CCNet, whatever) compiles the code, and packages it into NuGet packages (I'll explain why below)
The NuGet packages are exposed via a NuGet feed to Octopus
A release manager commands Octopus to deploy a "release" of your application to an environment (staging, production, etc.)
Octopus assembles all the NuGet packages, and pushes them to a "Tentacle", a small Windows Service that runs on your UAT/staging/production machines.
The Tentacle agent deploys the NuGet package, and configures it (see below)

Concepts

Octopus understands a few simple concepts:

Package: A NuGet package containing the smallest unit of deployment for part of your solution (for example, YourApp.Web, YourApp.Database)
Project: A group of packages and deployment steps, for example, YourApp
Release: A set of packages, with versions, for a project. For example, YourApp release 1.5 contains packages YourApp.Web 1.5.3 and YourApp.Database 1.5.1
Machine: A computer running the Tentacle service, that you intend to deploy packages to
Environment: A collection of computers, such as Staging or Production

Most of these concepts should be pretty intuitive from the UI - my goal is to make the release process pleasurable. Check out some of the screenshots to see where it's going.

Convention over configuration

Octopus was inspired by AppHarbor, which performs automated deployment of .NET applications from Git/Mercurial check-ins. AppHarbor has a very clever system for dealing with deployment:

If your VS solution has a Web project, it deploys it to an IIS site
If your web.config file has AppSettings with names matching variables you declare in the UI, they are automatically replaced

The end result is you never need to write automated deployment scripts for AppHarbor applications - it's automatic, because most applications work the same way.

Octopus shamelessly ~~steals~~ "borrows" these ideas, making enterprise application deployment as easy as cloud deployment on AppHarbor.

Conventions in Octopus include:

Replacing appSettings and configurationStrings entries
Updating the path of IIS websites if your package contains a web.config
Running any System.Configuration.Install.Installers (for performance counters, event logs and Windows Services)
Invoking a "Deploy.ps1" PowerShell script, if your package contains one

The cool thing is that all of these steps are run locally on the target server; you don't need to mess around with configuring WinRM or PowerShell Remoting.

NuGet packages

Instead of MSI's, Zip files or WebDeploy packages, Octopus standardizes on the NuGet package format. There are a few reasons for this:

Packaging files into a NuGet package is extremely easy
NuGet packages can be consumed via a feed, so other applications can easily query the available packages
CI tools like TFS and TeamCity will soon have support to make publishing NuGet packages easy

At first this might seem like a strange use of NuGet. Currently, NuGet is mostly used for packaging DLL's and other files, mostly for open source components. Why use NuGet for packaging complete web applications or Windows Services?

If you spend time in the world of Linux however, you can see that package managers (like NuGet) have evolved beyond libraries. If you want to install Apache or MySQL on Linux, you'll use a package manager. Shouldn't we be able to do the same on Windows?

An aspect of this idea that I think is really interesting is the concept of an internal "enterprise software inventory". Imagine if every version of every internal application you built was available from a NuGet feed, instead of a mix of File Shares, TFS Drops and TeamCity artifacts?

Reports and metrics

Since Octopus takes a pretty holistic view of your projects, environments and releases, it can be used to answer some interesting questions:

How long on average does it take Project A to get from Staging to Production?
How often do Staging releases fail user acceptance testing before going to Production?
Which version of Project A is in UAT right now?
What were the release notes of version 1.4 of Project A?

The business

I've spent a lot of time over the past few years working on open source projects. Open source projects are hard. Getting contributors is tricky, and even when you do, they usually focus on contributing "fun" features rather than bug fixing and documentation. Unless your project is large enough to have a lot of momentum, it's hard to attract committed contributors to do the not-so-fun things.

I think Octopus stands a much bigger chance of succeeding if it has people committed to doing the not-so-fun things, producing a very useful, very high-quality product that you can trust to reliably deploy your software. For this reason (plus, well, greed :-)) Octopus is going to be a commercial product, and my first MicroISV. Even if it's a colossal commercial failure, I'm looking forward to the experience of product development.

Pricing

At this stage I expect Octopus will be priced using a TeamCity model - free enough that you can use it to deploy one project to your organization, but by the time you need to deploy your second project, you'll have gotten enough value from it that you won't mind sharing the love.

What do you think?

This is one of my longer blog posts, and I hope I've done a good job of explaining what Octopus is all about. I'd like feedback, good or bad, about the direction. Specifically:

What do you think about the conventions? Can you think of any others? I've documented them a little better on the knowledge base.
Is the system described above something that you could see yourself/your organization using?
How are automated releases currently done within your organization?
What additional reports/metrics would be useful?

Google introduceert papieren e-mail

2011-06-09T15:15:09Z

Nieuw medium omzeilt Deep Packet Inspection

Inholland biedt afgestudeerden 4-jarige cursus aan

2011-05-16T11:50:08Z

Terpstra: “Het betreft een volledige compensatie van de eerdere studie.”

NuGet Package of the Week #7 - ELMAH (Error Logging Modules and Handlers) with SQL Server Compact

2011-05-23T21:03:33Z

Hey, have you implemented the NuGet Action Plan? Get on it, it'll take only 5 minutes: NuGet Action Plan - Upgrade to 1.3, Setup Automatic Updates, Get NuGet Package Explorer. NuGet 1.3 is out, so make sure you're set to automatically update!

The Backstory: I was thinking since the NuGet .NET package management site is starting to fill up that I should start looking for gems (no pun intended) in there. You know, really useful stuff that folks might otherwise not find. I'll look for mostly open source projects, ones I think are really useful. I'll look at how they built their NuGet packages, if there's anything interesting about the way the designed the out of the box experience (and anything they could do to make it better) as well as what the package itself does.

This week's Package of the Week is "ELMAH 1.2 with SQL Compact."

You may already use ELMAH. Don't stop reading now, because you're going to need this information!

Now, I've been blogging and promoting ELMAH (Error Logging Modules and Handlers) for years. It's the first thing I add to any ASP.NET project of any importance. It's one of the great little gems of ASP.NET open source.

I've shown "install-package elmah" in a number of talks and videos as my go-to demo for how NuGet and a well structured OSS package get you started quickly. Recently the ELMAH team has started to do some restructuring of their packages, and I wanted to showcase their work so you can not only learn from it, but also implement similar structures in your own packages.

First, spend a few minutes checking out how ELMAH works and how it can add to your ASP.NET project. Now, let's look at the (currently three, soon to be more) ELMAH packages that are up on NuGet.org.

There's currently:

ELMAH Core Library (no config) - "Core library for ELMAH (Error Logging Modules and Handlers) without any configuration"
- This one includes just the library. If you already have a custom ELMAH configuration or you want to create a new one, you can use and depend on this "core" package.
ELMAH - "ELMAH with initial configuration for getting started quickly. ELMAH (Error Logging Modules and Handlers) is an application-wide error logging facility that is completely pluggable. It can be dynamically added to a running ASP.NET web application, or even all ASP.NET web applications on a machine, without any need for re-compilation or re-deployment."
- This one includes the basics you'll need to for a drop-in/just-works configuration. This isn't the "core" library, but rather one that depends on the core. It's the "getting started" one.
ELMAH on MS SQL Server Compact - "ELMAH with configuration for getting started quickly on a Microsoft SQL Server Compact database as the error log."
- This package depends on the Elmah library and Elmah.CoreLibrary package as well as SqlServerCompact and adds configuration for using them together.
- Dependency chain: elmah.sqlservercompact -> elmah -> elmah.corelibrary

Here's the web.config.transform that elmah.sqlservercompact adds to your base ELMAH web.config:

<configuration>
  <elmah>  
    <errorLogtype="Elmah.SqlServerCompactErrorLog, Elmah"connectionStringName="elmah-sqlservercompact"/>
  </elmah>
  <connectionStrings>    
    <addname="elmah-sqlservercompact"connectionString="Data Source=|DataDirectory|\Elmah.sdf"/>
  </connectionStrings>
</configuration>

Give it a try, it's easy, just install-package elmah.sqlservercompact. This basic package dependency structure will enable the Elmah guys to create "quick starts" with different configurations that each depend on elmah, which depends on the elmah.corelibrary.

ELMAH supports many formats for storing errors, but I think that SQL Server Compact is a great choice for small to medium size applications.

Everything in one place and queryable.
No mess of text files or XML files
Database survives application restarts/recycles

If you're using a WebFarm, you might want to use a central SQL Server instance, but I know a number of folks who run WebFarms and still prefer their errors on a per WebServer basis.

When your application starts up, ELMAH will put the Elmah.sdf in your ~\App_Data folder. The database is structured like this image at right. With the SQL Server Compact tooling that's added with Visual Studio 2010 SP1 you can open the SDF file right in VS and query it if you like.

If you prefer a different backend for your logs, we should soon see NuGet "quick start" packages for many of the supported ELMAH storage solutions. You can even get your errors as RSS or JSON!

Microsoft SQL Server
Oracle (OracleErrorLog)
SQLite (version 3) database file
Microsoft Access (AccessErrorLog)
Loose XML files
RAM (in-memory)
SQL Server Compact Edition
MySQL
PostgreSQL

Also, be aware that Elmah has just be updated to version 1.2 with a number of new features and fixes. ELMAH works in .NET 1.1 and .NET 2, so there's no excuse for not checking it out!

Enjoy, and thanks to the ELMAH team for putting up with my nagging, bug reports and reviews over the last few months! It's only because I love the project so much!

Developer Stand up Comedy - Coding 4 Fun

2011-05-05T20:15:18Z

There aren't enough funny developer talks in my opinion. Lots of you folks are so funny in person and when I got to conferences or just have lunch we laugh and laugh. As with all niches or cliques, we're all part of a little club of inside jokes and shared stories.

It's always weird to get lots of comments on my technical talks. Many are positive, but sometimes I'll get one like "your not funy!" or "stop joking and start coding." Well, I realize my brand of edutainment isn't for everyone.

That said, this talk is happily content-free. If you hate my jokes, you may ignore it. ;)

Many years ago I did some local open mics and stand up, although I've never really put together a tight hour of material. I'm less of a stand up and more of a storyteller. Anyway, I got signed up for a talk in Holland last week called "Coding 4 Fun" but didn't prepare anything. I was already signed up for 6 other talks and had already presented 3 times by the time I showed up on stage at the 7pm "geek night" session.

There were a bunch of people there almost an hour early so I just sat on the end of the stage and we chatted. It was great fun. Next thing we knew, the guy was recording the video and the room was filling up.

So, I talked and told stories and chatted my co-workers and friends on IM for a while, the showed some community built NuGet packages. The only think I wish is that you could see the audience in this video. They were fun and engaged and silly.

Big thanks to Glenn Block and Clemens Vasters for sitting in the front row and playing along, and to Damian Edwards, Tim Heuer and Phil Haack for being silly over IM.

A lot of the jokes are in the IM or the code, so if you can't see them (even though I zoom in) with smooth streaming, download a higher res version so you can see them.

Coding for Fun with Scott Hanselman recorded in Holland in April of 2011

High Quality WMV (PC, XBox, MCE)
Medium Quality WMV (Lo-band, Mobile)
High Quality MP4 (iPad, WP7)
MP4 (iPod, Zune HD)

I hope you enjoy. Maybe one day I will put together a real act of "stand up for developers."

(Image)

2011-04-26T20:20:46Z

Working with the Chaos Monkey

2011-04-25T10:53:48Z

Late last year, the Netflix Tech Blog wrote about five lessons they learned moving to Amazon Web Services. AWS is, of course, the preeminent provider of so-called "cloud computing", so this can essentially be read as key advice for any website considering a move to the cloud. And it's great advice, too. Here's the one bit that struck me as most essential:

We’ve sometimes referred to the Netflix software architecture in AWS as our Rambo Architecture. Each system has to be able to succeed, no matter what, even all on its own. We’re designing each distributed system to expect and tolerate failure from other systems on which it depends.
If our recommendations system is down, we degrade the quality of our responses to our customers, but we still respond. We’ll show popular titles instead of personalized picks. If our search system is intolerably slow, streaming should still work perfectly fine.
One of the first systems our engineers built in AWS is called the Chaos Monkey. The Chaos Monkey’s job is to randomly kill instances and services within our architecture. If we aren’t constantly testing our ability to succeed despite failure, then it isn’t likely to work when it matters most – in the event of an unexpected outage.

Which, let's face it, seems like insane advice at first glance. I'm not sure many companies even understand why this would be a good idea, much less have the guts to attempt it. Raise your hand if where you work, someone deployed a daemon or service that randomly kills servers and processes in your server farm.

Now raise your other hand if that person is still employed by your company.

Who in their right mind would willingly choose to work with a Chaos Monkey?

Sometimes you don't get a choice; the Chaos Monkey chooses you. At Stack Exchange, we struggled for months with a bizarre problem. Every few days, one of the servers in the Oregon web farm would simply stop responding to all external network requests. No reason, no rationale, and no recovery except for a slow, excruciating shutdown sequence requiring the server to bluescreen before it would reboot.

We spent months -- literally months -- chasing this problem down. We walked the list of everything we could think of to solve it, and then some:

swapping network ports
replacing network cables
a different switch
multiple versions of the network driver
tweaking OS and driver level network settings
simplifying our network configuration and removing TProxy for more traditional X-FORWARDED-FOR
switching virtualization providers
changing our TCP/IP host model
getting Kernel hotfixes and applying them
involving high-level vendor support teams
some other stuff that I've now forgotten because I blacked out from the pain

At one point in this saga our team almost came to blows because we were so frustrated. (Well, as close to "blows" as a remote team can get over Skype, but you know what I mean.) Can you blame us? Every few days, one of our servers -- no telling which one -- would randomly wink off the network. The Chaos Monkey strikes again!

Even in our time of greatest frustration, I realized that there was a positive side to all this:

Where we had one server performing an essential function, we switched to two.
If we didn't have a sensible fallback for something, we created one.
We removed dependencies all over the place, paring down to the absolute minimum we required to run.
We implemented workarounds to stay running at all times, even when services we previously considered essential were suddenly no longer available.

Every week that went by, we made our system a tiny bit more redundant, because we had to. Despite the ongoing pain, it became clear that Chaos Monkey was actually doing us a big favor by forcing us to become extremely resilient. Not tomorrow, not someday, not at some indeterminate "we'll get to it eventually" point in the future, but right now where it hurts.

Now, none of this is new news; our problem is long since solved, and the Netflix Tech Blog article I'm referring to was posted last year. I've been meaning to write about it, but I've been a little busy. Maybe the timing is prophetic; AWS had a huge multi-day outage last week, which took several major websites down, along with a constellation of smaller sites.

Notably absent from that list of affected AWS sites? Netflix.

When you work with the Chaos Monkey, you quickly learn that everything happens for a reason. Except for those things which happen completely randomly. And that's why, even though it sounds crazy, the best way to avoid failure is to fail constantly.

Working with SSL at Development Time is easier with IISExpress

2011-04-21T22:59:06Z

One of the demos in my Mix 11 talk "An Overview of the MS Web Stack of Love" was showing how IIS Express and Visual Studio SP1 (as well as WebMatrix) can make working with SSL (Secure Sockets Layer) a heck of a lot easier.

If you've used Cassini before (that's the little built in Visual Web Developer Server) you've likely noticed that I doesn't support SSL. This makes working with real world sites a little challenging. If you want your Login pages and Account Management pages to use secure sockets, you'd typically have to do all your work with the full version of IIS, either installed on your own machine or using a shared server.

Here's a few ways to enable SSL. The first is new in Visual Studio 2010 SP1 and will allow you to use SSL on local host over ports 44300 and higher. This means you'll be able to test and develop how your site will work over SSL, but not over port 443 proper. I'll show you that in the final step.

If you watch the Mix video, you'll see towards the end where Damian Edwards educates me on this new SSL feature in VS2010SP1. I didn't know that VS2010SP1 (WebMatrix does also) installs some self-signed certificates and includes and option for turning on their use. However, as I pointed out in the video, that's only for high "strange" ports like 44300+, so my more complex example still has value if you want standard port numbers.

The Easy Way - Local SSL with IIS Express and VS2010 or WebMatrix

If you have IIS Express and VS2010SP1, you can do this now and follow along. Make a new ASP.NET Site in Visual Studio.

Right click on the Project in Solution and select Use IIS Express. You can also set IIS Express as the default from the Tools | Options | Projects and Solutions | Web Projects.

Next, click Yes, and VS will "make a new site" on IIS Express. What does that mean?

Click yes and let's find out.

Remember that IIS Express is really IIS. It's just "local personal not-a-service" IIS. That means that IISExpress puts its config files in C:\Users\YOU\Documents\IISExpress\config rather than in some machine-wide location.

My project is called "MvcApplication18" so I can open up my ApplicationHost.config and look for "MvcApplication18." You can too. There's my site, right there, in IISExpress's applicationHost.config:

<site name="MvcApplication18" id="39">
    <application path="/" applicationPool="Clr4IntegratedAppPool">
        <virtualDirectory path="/" physicalPath="c:\users\scottha\documents\visual studio 2010\Projects\MvcApplication18\MvcApplication18" />
    </application>
    <bindings>
        <binding protocol="http" bindingInformation="*:15408:localhost" />
    </bindings>
</site>

Note the binding section. I can see that my site will show up on http://localhost:15408.

Go back to Visual Studio, click on your project and press F4 to bring up the properties dialog. You can also press Ctrl-W, then P, or select View | Property Window.

Since I'm using IIS Express and I have VS2010 SP1 installed, I have a new option, "SSL Enabled." If I click it, a new "SSL URL" shows up with a new port number chosen from that pool of ports I mentioned before.

Go back over to your ApplicationHost.config if you want to see what really happened.

<site name="MvcApplication18" id="39">
<application path="/" applicationPool="Clr4IntegratedAppPool">
    <virtualDirectory path="/" physicalPath="c:\users\scottha\documents\visual studio 2010\Projects\MvcApplication18\MvcApplication18" />
</application>
<bindings>
    <binding protocol="http" bindingInformation="*:15408:localhost" />
    <binding protocol="https" bindingInformation="*:44302:localhost" />
</bindings>

See that new binding? That was created for us when we clicked SSL Enabled = True.

Run your site. Visit it with and without SSL. Don't forget the port number! You're now running under SSL locally, but you're reminded you are a bad person because this certificate is not trusted. Still, create an app, check a box and you've got local SSL.

Ok, how can we get this running in a slightly better way? I want:

A friendly machine name, not localhost.
People to be able to talk to my instance of IIS Express from the outside.
Actual SSL over port 443.
My ASP.NET application to switch between SSL and not automatically when I'm logging in.
My self-signed certificate to be trusted so I don't get warnings.
To use PowerShell at some point for no reason at all because that's bad-ass.

Here we go.

The Hard Ninja Way - Local SSL over 443 with IIS Express and the Gracious Manatee that is The Command Line

These steps may seem a little scary, but it's useful to know that they are happening (or have happened) already to make the Easy Way work for you. I'll show you how to do it yourself, then I'll show you an undocumented way to make part of The Hard Way even easier. It's important to know what's happening though and why when you start running random commands from an Administrator Command Prompt, right?

1. Getting IIS Express to serve externally over Port 80

My machine is called HANSELMAN-W500, so I'll use that name. You could update your hosts file and use a friendly name. To start, use your computer name. if you don't know the name of your computer, you're silly. Go to the command prompt and type "HOSTNAME" to find out.

First, we need to tell HTTP.SYS at the kernel level that it's OK to let everyone talk to this URL by making an "Url Reservation." From an administrative command prompt:

netsh http add urlacl url=http://hanselman-w500:80/ user=everyone

Next, as I want to be able to talk to IIS Express from outside (folks on my network, etc. Not just localhost) then I need to allow IIS Express through the Windows Firewall. I can do that graphically from Windows, or type:

netsh firewall add portopening TCP 80 IISExpressWeb enable ALL

Finally, I need to make sure that my project will use Port 80. I can do that one of two ways. I can either edit the applicationHost.config manually and add the binding (my recommended way):

<site name="MvcApplication18" id="39">
    <application path="/" applicationPool="Clr4IntegratedAppPool">
        <virtualDirectory path="/" physicalPath="c:\users\scottha\documents\visual studio 2010\Projects\MvcApplication18\MvcApplication18" />
    </application>
    <bindings>
        <binding protocol="http" bindingInformation="*:15408:localhost" />
        <binding protocol="https" bindingInformation="*:44302:localhost" />
        <binding protocol="http" bindingInformation="*:80:hanselman-w500" />
    </bindings>
</site>

Or, I can do that from the command line too! Although it's a little scary. I can confirm my changes in ApplicationHost.config though if I mess up.

"c:\Program Files (x86)\IIS Express\appcmd.exe" set site /site.name:MvcApplication18 /+bindings.[protocol='http',bindingInformation='*:80:hanselman-w500']

Notice that I'm using the appcmd.exe that came with IIS Express. I don't want to mess up my actual IIS installation if I have one.

2. Making an SSL Cert, hooking it up to IIS Express and making it Trusted

Let's make a SSL certificate of our own. Note the CN=. I'm making it my Computer Name, but you could make it nerddinner.com or whatever makes you happy. It should line up with whatever name you've been using so far.

makecert -r -pe -n "CN=HANSELMAN-W500" -b 01/01/2000 -e 01/01/2036 -eku 1.3.6.1.5.5.7.3.1 -ss my -sr localMachine -sky exchange -sp "Microsoft RSA SChannel Cryptographic Provider" -sy 12

Now, a tricky part. Go find this certificate in the Certificate Manager. Run MMC.exe, go File | Add/Remove Snap In, then select Certificates. Pick the Computer Account. (This is why you can't just run certmgr.msc) and add it.

It'll likely be the certificate with an expiration data of 1/1/2036 under Personal Certificates. Double click on your certificate. Go to Details, and scroll down to Thumbprint. Copy that into the clipboard, as that identifies our new certificate.

Remove all the spaces from that Thumbprint hash. You can remove those spaces with Notepad if you're Phil Haack, or in PowerShell if you're awesome:

C:\>"41 d8 50 95 11 0e 1d f6 8c 89 84 97 55 25 a8 60 59 35 23 0a" -replace " "
41d85095110e1df68c8984975525a8605935230a

Take the hash and plug it in to the end of THIS command:

netsh http add sslcert ipport=0.0.0.0:443 appid={214124cd-d05b-4309-9af9-9caa44b2b74a} certhash=YOURCERTHASHHERE

The AppId doesn't really matter, its just a GUID. This tells HTTP.SYS that we're using that certificate. Leave the Certificate Manager MMC running.

Now, tell HTTP.SYS that we're cool with port 443 also (we told it that 80 was cool a minute ago, remember?):

netsh http add urlacl url=https://hanselman-w500:443/ user=Everyone

Return to your applicationHost.config and add the 443 binding for your site:

<site name="MvcApplication18" id="39">
  <application path="/" applicationPool="Clr4IntegratedAppPool">
     <virtualDirectory path="/" physicalPath="c:\users\scottha\documents\visual studio 2010\Projects\MvcApplication18\MvcApplication18" />
  </application>
  <bindings>
    <binding protocol="http" bindingInformation="*:15408:localhost" />
    <binding protocol="https" bindingInformation="*:44302:localhost" />
    <binding protocol="http" bindingInformation="*:80:hanselman-w500" />
    <binding protocol="https" bindingInformation="*:443:hanselman-w500" />
  </bindings>
</site>

If I say "Show All Sites" from the IIS Express tray icon, I'll see my site(s) and the URLs they are bound to.

Now I can visit https://hanselman-w500, but again I get a certificate error.

Go back to the CertMgr MMC, and drag your self-signed SSL Certificate from Personal into Trusted Root Certificates.

Suddenly my local SSD site is legit! Very cool.

3. Getting ASP.NET to force SSL with an URL Rewrite Rule

One of the things Cassini (Visual Studio Web Developer Server) can't do is UrlRewriting. I want my app to force SSL when I hit /account/logon or /account/register. I'll add this to the first node of system.webServer in my app's web.config:

<rewrite>
  <rules>
    <rule name="Redirect to HTTPS" stopProcessing="true">
      <match url="^account/logon$|^account/register$" />
      <conditions>
        <add input="{HTTPS}" pattern="^OFF$" />
      </conditions>
      <action type="Redirect" url="https://{HTTP_HOST}/{R:0}" redirectType="Permanent" />
    </rule>
  </rules>
</rewrite>

I could also use the RequireHttps attribute on my controllers if I like.

Appendix Z: A totally undocumented way to make part of this easier that you use at your own risk

There's a command line helper deep inside of the IIS Express directory that I never mentioned to you. We never spoke! I don't know you. Who is this? Stop calling! ;)

C:\Program Files (x86)\IIS Express>IisExpressAdminCmd.exe
Usage: iisexpressadmincmd.exe <command> <parameters>
Supported commands:
      setupFriendlyHostnameUrl -url:<url>
      deleteFriendlyHostnameUrl -url:<url>
      setupUrl -url:<url>
      deleteUrl -url:<url>
      setupSslUrl -url:<url> -CertHash:<value>
      setupSslUrl -url:<url> -UseSelfSigned
      deleteSslUrl -url:<url>

Examples:
1) Configure "http.sys" and "hosts" file for friendly hostname "contoso":
iisexpressadmincmd setupFriendlyHostnameUrl -url:http://contoso:80/
2) Remove "http.sys" configuration and "hosts" file entry for the friendly 
hostname "contoso":
iisexpressadmincmd deleteFriendlyHostnameUrl -url:http://contoso:80/

From the command line with this utility, I can quickly setup my hosts file and my HTTP.SYS Url ACLs with one command:

C:\Program Files (x86)\IIS Express>IisExpressAdminCmd.exe setupFriendlyHostnameU
rl -url:http://daddyisawesome:80/
Command 'setupFriendlyHostnameUrl' completed.

And remove them:

C:\Program Files (x86)\IIS Express>IisExpressAdminCmd.exe deleteFriendlyHostname
Url -url:http://daddyisawesome:80/
Command 'deleteFriendlyHostnameUrl' completed.

At this point you just need to update the IISExpress applicationHost.config with the correct binding. You can also use IISExpressAdminCmd setupSslUrl with SSL ports that are already reserved. However, I really think The Hard Way is best because you can really see what's going on, and you have more control.

Make It Stop!

How do I undo it all? Delete the Certificate in CertMgr, and from an Administration Console:

netsh http delete sslcert ipport=0.0.0.0:443
netsh http delete urlacl url=http://hanselman-w500:80/
netsh http delete urlacl url=https://hanselman-w500:443/

If you have existing SSLCerts registered with HTTP.sys, the adjust these commands.

Enjoy! Thanks to CarlosAG for his help with this post.