Kevin Dietz's Blog

Book Notes - The Greatest Show on Earth, Chapter 3

2011-01-03T15:11:53.201625-06:00

The Greatest Show on Earth

a book by Richard Dawkins

Notes on Chapter 3: The Primrose Path to Macro-Evolution

Like human-influenced artificial selection, the history of flowers also teaches us about evolution
Wild roses were (genetically) sculpted by the human eye into the varieties we have today, but insects were the first domesticators of flowers
Russians bred giant sunflowers, before that sunflowers had already been cultivated by native Americans, which made them larger than wild sunflowers, and before that, all brightly colored flowers owed their existence to insects (and other domesticators, such as birds, etc.)
To take advantage of sexual reproduction, flowers need cross-fertilization
Many plants use wind as a mechanism for pollen transportation, but this method requires a large production of pollen
Plants use nectar as a bribe for pollen transportation
n Madagascar, there is a long-tubuled orchid which prompted them to predict there existed a moth with a tongue long enough to reach the nectar. Such moth was later discovered.
Insects see the color red poorly ... most red flowers we see are pollinated by birds, not insects
nsects sight is shifted toward the ultraviolent spectrum, which humans can't see
The color of flowers has been shaped by the eyes of insects
Insects and flowers have a symbiotic relationship. Flowers breed insects to be good transporters of pollen. Insects breed flowers to be reliable producers of food. They therefore co-evolve together.
Another example of natural domesticators: brightly colored male birds, caused by 'sexual selection'. Females choose males which they find most attractive. Females remain plainer and more naturally camouflaged.
The heikea japonica crab shell has a pattern which bears the resemblance of a samurai warrior. Carl Sagan thought it was caused by human selection (fishermen threw back the crabs with the most resemblance). Dawkins thinks it is just a coincidence.
There is a caterpillar with a tail that looks like a snake
Deep sea angler fish use bioluminescent tail as bait to catch fish; in a sense prey fish "choose" (inadvertantly, of course) which angler fish survive
Choosing agents are unnecessary ... survival itself acts as the selector. All other examples previously given are simply special cases of the general phenomenon
Discussion of how we can test evolution by experimentation
Since 1896, agronomists have experimentally create two lines of maize, one with very high oil content, and one with very low content, both starting with the same variety from 1896
Rats have been experimentally bred in the lab to be more resistant to tooth decay
A good discussion of evolutionary tradeoffs follows. Example, antelopes with longer legs run faster, but are more prone to fracture. Life forms reach equilibriums based on tradeoffs needed.
Foxes have been experimentally bred to be tamer
A discussion of various tricks flowers use to fool insects into transporting their pollen
Various experiments have shown that dramatic changes can occur in very short timeframes; nature can do the same work, but needs more time; has there been enough time for evolution to work? This question will be addressed in Chapter 4.

Kevin's Commentary on Chapter 3

In this chapter Dawkins shows us that human eyes are not the only eyes that act as selecting agents. Insects select brightly colored flowers by choosing them for their attractiveness and their nectar, and female birds choose male birds they find attractive, thus causing males to become more brightly colored over time. Finally, Dawkins reveals the full lesson of evolution, that you don't even need eyes, human or otherwise, to act as the selecting agent, that mere survival will do the job. And that is what evolution by natural selection is all about.

I understand his argumentation fully. Unfortunately, I think the argument will do nothing to convince those that are determined to deny evolution. I've heard creationists use some of these same facts as evidence for their point-of-view. For example, I've heard an argument that says the existence of vanilla orchids and vanilla bees proves an act of special creation. Since one cannot exist without the other, it must be that a deity created both simultaneously. Such a fact is easily explainable via co-evolution, but that is even harder for people to fathom.

The experiments discussed are compelling, so I hope that, dare I say, ahem, "non-believers", will be persuaded by them.

Book Notes - The Greatest Show on Earth, Chapter 2

2010-12-15T22:39:56.659625-06:00

The Greatest Show on Earth - Chapter 2

a book by Richard Dawkins

Summary of Chapter 2: Dogs, Cows, and Cabbages

Why did it take humans so long to discover evolution?
Ernst Mayr speculate it was due to Plato essentialism, what we see is a shadow of true reality. For example, a triangle drawn in the sand is just a shadow of a true, essential triangle
In essentialism thinking, there is a "standard rabbit", and all rabbits we see are statistical variations on the standard
Population thinking is the antithesis of essentialism ... there is no standard rabbit
All life forms are linked to all other life forms via a chain of intermediates
Thought experiment ... think of a rabbit, then imagine a chain of ancestors, each link in the chain looking nearly identical, but changing slowly. Mentally traverse back to the common ancestor of rabbits and leopards. Perhaps the animal look a little like a shrew, but certainly looked nothing like either a rabbit or leopard. Now make the hairpin turn and begin walking forward in time, this time taking the path the leads to leopards
Note there is never a direct cross between a rabbit and a leopard
"Modern species do not evolve into other modern species, they just share ancestors: they are cousins"
Humans have very successfully domesticated plants and animals via artificial selection
The wild cabbage has been turned into broccoli, cauliflower, kohlrabi, kale, brussel sprouts, spring greens, and romanescu
The wolf has been turned into two hundred breeds of dogs
All domestic dogs are descended from wolves, not jackals, not coyotes, and not foxes
What we are really doing is sculpting gene pools
Gregor Mendel is the father of genetics
Genes are shuffled between generations
Darwin came very close to discovering Mendel's law of non-blending genes
Geographic barriers inhibit gene shuffling. For example, it will take time for the genes of a rat in Madrid to migrate all the way over to rat populations in Siberia
Evolution occurs when there is a systematic increase or decrease of a gene within a gene pool
A lengthy discussion of dog breeding follows
Overly muscular cows and pigs are bred by disabling a gene that makes myostatin
Although morally problematic, there seems little doubt that eugenic breeding of humans could produce superior body-builders, pearl fishers, musicians, poets, etc.
Next, there's a discussion of computer simulations that can derive new forms based on a human selecting desirable features on each generation, but without an overall master plan or design
Darwin also discussed artificial selection extensively in Chapter 1 of On the Origin of Species
Via non-random survival, natural selection over a longer period of time can produce that same affects as artificial selection can in a short timeframe

Kevin's Commentary on Chapter 2

As Dawkins points out in the last paragraph of the chapter, Charles Darwin's first argument in Chapter 1 of On the Origin of Species, Variation Under Domestication, discusses artificial selection that occurs as humans domesticate plants and animals to better serve our purposes. Thus, it's no surprise that Dawkins would devote his first chapter to the same topic.

The basic argument here is that artificial selection proves that descent with modification can happen, at least under special circumstances. Thus, we know that it's at least possible. It doesn't violate any laws of physics, and therefore doesn't require, at least in principle, supernatural guidance.

The thought experiment that Dawkins walks us through, showing how rabbits and leopards are linked via common ancestry, is fascinating to think about. Dawkins believes that there is sufficient evidence to say that all living creatures are descended from a single ancestor. If that's true, then any two organisms living today, no matter how different from each other, can be linked together via their ancestry and the hairpin turn. The greater the difference between the two organisms, the farther back in time one will have to go to find the common ancestor that defines the hairpin turn in the thought experiment. Try doing this thought experiment yourself using two organisms of your choosing. When I do it, I try to imagine what the common ancestor might have looked like.

Darwin himself commented on the temptation of trying to imagine an intermediate between two modern species. It's natural for us to want to do that, and creationists seem to seize upon this as an opportunity to lead the unwary astray. It is absolutely critical to remember that modern species don't evolve into other modern species. There is no rabbit-leopard cross. The common ancestor looked nothing like either. It might have looked a bit like a shrew, but even then, probably not a modern shrew.

Book Notes - The Greatest Show on Earth, Chapter 1

2010-12-12T23:14:06.987-06:00

The Greatest Show on Earth - Chapter 1

a book by Richard Dawkins

Summary of Chapter 1: Only a Theory?

Dawkins asks us to imagine we are a teacher of Roman history but you were distracted by "ignoramuses" who denied Romans ever exists. You were demanded to "teach the controversy" ... how frustrating
Things like this really happen, as is the case with Holocaust-deniers
Science teachers today face similar opposition w.r.t. teaching evolution
Many religious leaders accept evolution and don't find that it conflicts with their faith.
Dawkins doesn't intend this as an anti-religious book, only a book that defends the evidence for evolution (he's ripped religion to shreds in other books, no reason to do it again here)
However, religious leaders accepting evolution doesn't seem to transfer to their congregations ... 40% of Americans deny that humans evolved, and instead were created by God in the last 10,000 years. He refers to this group as "history deniers", or "40 percenters"
Perhaps this is true because clergy talk a lot in symbolism, and don't make it clear what's symbolic and what's literal. How is the congregation supposed to know the difference?
Evolution is a fact beyond reasonable, serious, sane, intelligent doubt
"No reputable scientist disputes it, and no unbiased reader will close the book doubting it"
Next, Dawkins spends time discussing the definition of a "theory", and why there is so much confusion around it, especially since people of faith love to apply the label "only a theory" to evolution
The definition of theory has 2 meanings 1) A set of ideas that explains a group of facts; a hypothesis confirmed by observation or experiment, etc. OR 2) A hypothesis, speculation, conjecture, etc.
He thinks scientists mean Sense 1 and creationists "mischievously" mean Sense 2
Evolution is a theory in the same sense as the theory that the Earth orbits the Sun (heliocentric theory)
He then discusses whether evolution has been or can be 'proved'. He discusses what it means to prove something. By a certain philosophy, only mathematicians can ever prove anything, such as the Pythagorean Theorem. By this view, scientists can never prove anything.
However, scientists can accumulate so much evidence for a theory that continuing to deny it a fact becomes ridiculous
When theories are beyond sensible doubt, we call them 'facts'
Even labeling something a 'fact' does not have the same rigorous status as a mathematical proof
He discusses the definition of a 'fact', and points out that even things we think are facts are not always reliable, such as the case with false-convictions based on eye-witness testimony
Evolution happens too slowly for eye-witness testimony, so scientists must use inference
"This book will take inference seriously - not mere inference but proper scientific inference"
Example of inference: Understanding continental drift, we can infer that South America was once joined to Africa
Evolution is both fact and theory ... all living things are cousins is a fact, the process that drives it, natural selection, is a theory (Sense 1 theory, not Sense 2)
In Darwin's day, evolution was more conjectural, but now we have compelling evidence that it is true
"Nowadays it is no longer possible to dispute the fact of evolution itself"
All reputable biologists agree that natural selection is one of the most important driving forces
Because we cannot directly observe evolution, we will examine the case for evolution much the way a detective examines a crime scene

Kevin's Commentary on Chapter 1

I have personally engaged in conversations and debates with people of faith, namely Christians, that attack or dismiss evolution as "only a theory." Therefore, I find Dawkins lengthy discussion of theories, hypotheses, facts, and proofs both useful and relevant to my experience.

I watched a YouTube video with Kenneth Miller not long ago, and I thought his explanations of theories vs. facts was as good, if not even better than Dawkin's treatment. Miller said that people generally think that facts are things that scientists are sure of, and theories are things we're not so sure of. But, he explains, theories are actually more powerful than facts because theories explain facts. I love that explanation. Dawkins says nearly the exact same thing when he quotes the dictionary definition of theory.

Evolution by natural selection is an elegant, powerful theory that explains an enormous variety of facts we've collected, such as the fossil evidence facts, the facts about cells and DNA, the fact that life is so diverse and different around the world, and the facts about the behaviors of different forms of life. Labeling evolution as a "theory" does not drag it down into the depths of doubt, but rather, lifts it up to a higher place in the world of science.

The creation hypothesis, on the other hand, does a terrible job of explaining the facts of the world around us (more on this in future chapters). The contrast between these two world views I find very compelling.

Book Notes - The Greatest Show on Earth

2010-12-12T23:04:31.222-06:00

The Greatest Show on Earth - Preface Notes

a book by Richard Dawkins

Synopsis: A book written to present the evidence for evolution

Summary of Preface:

· Evidence for evolution grows every day, yet paradoxically opposition is stronger than ever
· The 'theory' of evolution is actually a fact
· Dawkins briefly mentions some of his earlier books related to evolution, but thinks none of them explicitly laid out the evidence for evolution, so he wrote this book
· He wrote a highly favorable review of Jerry Coyne's "Why Evolution is True"
· Considered using "Only a Theory" as the title for the book, but Kenneth Miller was already using it, and settled on the Greatest Show On Earth, with Only a Theory? (emphasis on the question mark) as the title of the first chapter

Kevin's Commentary on the Preface

The canonical source for the evidence of evolution is, of course, from Charles Darwin himself, "On The Origin of Species", and yes, I've actually read it. However, the Victorian prose makes it a difficult read for many Americans, and therefore it always helps to have modern interpretations. Also, there has been a lot more tangible evidence discovered since the original publishing of "On The Origin of Species" (1859), and it's therefore useful to augment Darwin's original arguments.

I've only read one other Dawkins book so far, The God Delusion, but I intend to eventually read all of Dawkins works.

I've read Jerry Coyne's "Why Evolution is True" and agree with Dawkins that it is excellent.

I've read Kenneth Miller's book, "Finding Darwin's God" and have also watched various debates and lectures of him on YouTube. His defense of evolution is superb, but as a Catholic he still ultimately, in the broadest sense, believes in a creator and accepts many aspects of the Christian faith, which is a bit of a paradox I have trouble understanding.

Why I Might Get an iPad

2010-02-04T01:02:57.258-06:00

As you might have gleaned from my last post, I'm torn on the new Apple iPad. Like so many naysayers that have come out over the last week, I'm disappointed in all the things lacking from the first generation iPad: multitasking, a camera, Flash support, Microsoft Exchange support (Apple is unclear about this, but excludes it from the list of supported e-mail providers on its web site), blah, blah, blah. Of course I'd love it if the iPad had all of these things, didn't run any slower because of them, and didn't cost any more either. We always want more. But it is what it is. I can't change Apple's engineering decisions. I can only decide whether the device is something worth buying or not.

So I've put my thinking cap on over the last week to try to imagine how I might use such a thing.

The first thing I imagine is an accessorized iPad, not a stock iPad. I've heard people complain about how the iPad has a slight curvature on the back that will make it difficult to put it on your lap and type on the virtual keyboard. Forget about that. We're going to put the iPad inside of a nice portfolio of some kind that will remove this objection. I want something to protect it a little bit if I handle it less than gently. Apple already has a nice looking iPad portfolio on their web site, and I imagine that there will be plenty of other third-party options just like there are zillions of cases for the iPhone. Imagine a nice portfolio case for the iPad. A place for a pen, maybe a stylus, a little pad of real paper, and a holder to stuff in a few business cards. The photos of the Apple portfolio doesn't show good enough photos of the inside of it, so I can't tell if it will have those features or not. But surely, there will be some really nice cases for it. The case will provide a nice typing surface for lap typing. I'm not worried about that objection.

Next, I imagine somebody is going to make a nice stylus that I can use with the iPad, and there will be some way of using the stylus to create hand-written notes. And I'm not talking about some little plastic stick, I mean a nice stylus, with the same quality and feel of a good Waterman pen. Such things already exist for other PDA's and smartphones, so surely there will be some kind of solution for the iPad. I'm not going to get too wrapped around the axle about the lack of note-taking capabilities, because surely this is going to be addressed some how, some way.

Those two accessories will give me a nice, portable platform for transporting the iPad with me. Would I meet my friend at Starbucks for tea and take something like that with me? Yeah, I might. If I didn't need my laptop, but still wanted to have a business-oriented meeting that might require opening a document or a web site, this wouldn't be a bad option. My 17 inch MacBook Pro, nice as it is, is still pretty big and heavy. I really don't want to take that to Starbucks unless I know for sure I really need it. Now to be sure, there would be plenty of times when I wouldn't take *either* the laptop or the iPad, I would just take my iPhone, but I can imagine there would be times where the larger screen of the iPad would come in really handy at Starbucks.

I'm also imagining, however, that I would use the iPad around the house much more frequently than I would take it to Starbucks.

My home is laid out in a long and narrow footprint. At one end is my office, and clear at the other end of the house is the master bedroom. In the evenings, my wife is usually laying in bed watching TV, perhaps using her 13 inch Macbook Pro to do a little work between commercials. I'm all the way at the other end of the house in my office, usually surfing the web or organizing my e-mail. That sucks. It would be nice to spend some time with her together. Now, me being a dude, my original solution to this dilemma was far different than getting a netbook, Macbook Air, or a tablet PC. My suggestion was to get a second Mac Pro, a second 30 inch monitor, install a heavy-duty wall-mounted monitor arm, and a wall-mounted sliding keyboard tray. Then, not only could I surf and check e-mail, I could do real work, like actual programming. My wife was so impressed with this idea that she said "no freakin' way."

Given where I'm at now, the obvious solution would be to simply take my 17 inch MacBook Pro into the bedroom. I've done it. It works. It's not a bad solution. It's a little bit bulkier and clumsier that I would like, however. If I'm typing a long e-mail response, the laptop keyboard is nice to have. But most of the time, I'm not doing much typing. It would be nice to have a smaller, lighter device that I could switch between landscape and portrait mode. The iPad does have some potential here.

I am disappointed that Apple hasn't specifically mentioned support for Microsoft Exchange. I use a hosted Exchange server to get my e-mail. One thing that would be nice to do laying in bed is to go through my Inbox and organize my e-mails. I can do this on my iPhone, but there's certain things I can't do. For example, if I want to create a new folder to move the e-mail into it, I can't do that on the iPhone. Or if I want to save off a PDF receipt onto my QuickBooks server, I can't do that. Typing anything more than one-line responses to an e-mail is too tedious on the iPhone. So the iPad could give me a better platform for doing that, and it would be a lot easier to flop around in bed with the iPad than the 17 inch laptop. Let's assume the iPad has no support for Exchange. That sucks, but I imagine that a future software update will solve that. In the meantime, I could use the web-based Outlook Web Access (OWA) from the iPad's browser and do it that way. Such a solution would be unthinkable on the iPhone, but with the larger screen of the iPad, it might be a good-enough transition solution that it could hold me over until there was native Exchange support. I don't really like the OWA interface, however, so I really hope that Exchange support is coming in the not-too-distant future.

Another thing I like to do a lot is type notes into my wiki. If I'm searching the web for a topic, and I find some good links, I typically copy-and-paste them into my wiki along with some contextual notes. I can access my wiki from Safari, and it doesn't use any Flash plug-ins, so I could see myself sitting on the couch or laying in bed, watching Celebrity Sex Rehab with Dr. Drew on TV, and typing away on my wiki with my iPad. I really do use my wiki a lot, as a journal, a to-do list, a software architecture platform, and lots of other things, so it would be kind of nice to have a light-weight device with good battery life that I could use to update my wiki when I'm not sitting at my desk. Ditto for writing blog entries as I'm doing now.

Lastly, and this may be the sleeper app that nobody has thought much about yet, the iPad might make a really great platform for remote desktop-ing into other computers around the house. In addition to my primary desktop and laptop, I've got a couple secondary desktop computers, 5 rack-mounted servers down in the basement, and numerous virtual machines for various applications. Being able to hit any of those from my iPad would be really cool. There are already several remote desktop apps available for the iPhone, but the iPhone's screen is really too small for serious work. I have to assume that the remote desktop vendors will update their apps to take advantage of the native screen resolution of the iPad.

One frequent use of remote desktop is accessing my QuickBooks installation. As I mentioned in an earlier blog post, I decided to set up a dedicated virtual machine specifically for running QuickBooks and storing finance-related documents (bank statements, receipts, etc.). That way I can access QuickBooks from either my desktop or my laptop, or any other computer for that matter. It's not the fastest solution in the world, but it's worked out fairly well. Like a lot of small business owners, I hate doing the books, and I procrastinate doing it for as long as possible. But if I could sit on the couch with my iPad, remote desk into the QuickBooks VM, enter a few invoices, pay a few bills, and reconcile a few bank statements, I might be more motivated to keep my financials up-to-date. People are complaining about the lack of multitasking, but with remote desktop, problem solved. Who knows, maybe remote desktop will turn out to be the killer app for the iPad.

Speaking of multitasking, I hear people complaining that they won't be able to run an app and listen to music at the same time. I don't know, but for my money, I'm probably not going to play music on the iPad. Photos, maybe. Watching a movie on an airplane, definitely. But if I want to run an app and listen to music at the same time, I'll just play the music on my iPhone or iPod Nano. I mean, I'm always going to carry my iPhone with me, iPad or no iPad, and an iPod Nano or Shuffle is so small that I could take one of those with me too if I really thought I needed it. Who cares if the iPad can't play music and run an app at the same time. Now, multitasking would be really nice, don't get me wrong ... I hope they add that in iPhone OS 4.0, but I think I could still do useful work on the iPad even without multitasking.

So that's how I could see myself using the iPad ... sitting on the couch surfing the web, updating my wiki, writing a blog, organizing my e-mail (please, please, please give me Exchange support), and catching up on QuickBooks via remote desktop. Let's see how close reality comes to that.

New Missing Link Discovered: The Apple iPad

2010-02-03T23:48:34.523625-06:00

Two years ago I began slowly switching to Apple hardware, even though I am primarily a Windows / .NET developer and thus require both Windows and Visual Studio. I'm using Macs because I think Apple makes great hardware. I currently own a Mac Pro, a 17 inch MacBook Pro, and an iPhone 3G.

The 17 inch MacBook Pro was my latest addition. Having a high-end desktop and a small smartphone, you'd think you might want something in between those two extremes. And whadaya know, that's a niche the laptop fills. I needed a fairly powerful laptop that I could use on the road for sales presentations, and with the MacBook Pro I can not only run Windows Server under a VM, but I can switch back to OS/X and use it for testing the client-side portion of my software on the Mac. Bonus.

So the laptop is my missing link, if you will, between my desktop and my smartphone. I like the evolution analogy. Creationists like to argue that evolution can't be true because there are missing links between species, but I love the evolutionists' response to that argument: there will always be missing links, no matter how many new species we discover, because if there are two species A and C, and a missing link species B is later discovered, all that does is create two more missing links, one from A -> B, and another from B -> C.

Thinking along these lines, a laptop that fills in the gap between my desktop and smartphone doesn't solve my problem, it only doubles it. Now I have to wonder what device should fill in the gap between my desktop and my laptop, and what device fills in the gap between my laptop and my smartphone.

This is the mindset that Steve Jobs put us in last week as he unveiled the iPad, Apple's long-awaited tablet computing device. Do we need such a thing, and if so, why?

Given that I already own three Apple devices, one for each major computing category, I'm of course interested in the iPad. At this point, I would be more prone to buy an iPad than I would a netbook. But I'd also be just as likely to keep the three devices I've got rather than adding a fourth device. For me, that is a key difference than it is for others that have ditched their desktops altogether. For me, the iPad, would be a fourth device, not a third device as Steve Jobs argued last week. I'll have more to say about the iPad in my next post.

This is the complete tree of computing technology as I see it. You start with a Mac Pro and an iPhone. To fill in the gap between the two, you get a 17 inch MacBook Pro. To fill in the gap between the Mac Pro and the 17 inch MacBook Pro, you get a 27 inch iMac. Now, you fill in the gap between the 27 inch iMac and the 17 inch MacBook Pro with a 21 inch iMac. Next, you fill in the gap between the 17 inch MacBook Pro and the iPhone with an iPad. We're not done yet. The gap from the iPad to the iPhone is filled in with an iPod Touch (which has more memory than an iPhone). Of course, you now have to fill the gap between the iPad and the 17 inch MacBook Pro with a 13 inch MacBook Pro. Then, of course a 15 incher fills in the gap between the 17 and 13 inchers. But now you still gotta fill in the gap between the 13 incher and the iPad, so you get yourself a MacBook Air. And finally, you fill in the gap between the MacBook Air and the iPad with a Asus T91 tablet netbook. Check back with me in a few months. Oh crap, then I still need to get a Kindle to fill the the gap between the iPad and the iPod Touch.

Demo-Driven Development

2009-05-03T17:35:12.861-05:00

I'm working my way through Bob Walsh's book, Micro-ISV.

The first chapter discusses an important topic that I've already blogged about ... how do you come up with an idea in the first place? This chapter outlines three basic approaches. First, there's the systematic approach, where you identify industries you're interested in, and keep drilling into it in finer grained research until you have a specific niche idea. Second, is the "Joel" approach (hey, that's the third blog now that involves Joel Spolsky). When asked how he got started, Joel said they had three product ideas, and FogBugz was the one closest to being ready to ship, so that's what they focused on. And third, is the "dammit" approach.

In the "dammit" approach you look for a significant source of frustration in your daily work. I've been working on my new product, MergeMagician, for a little over a month now. Of these three approaches, I think the "dammit" approach best describes how I decided on MergeMagician. I've been developing software configuration management tools for a long time now, first in defect tracking, then build management, so I understand this area well, and there are a lot of opportunities for "dammit" discovery within this domain.

The MergeMagician "dammit" is that branching and continuous integration don't work well together. I want to be able to work on features independently of each other, so then when one feature is done, I can ship the product with the new feature. I never know how long each feature will take, so it would be nice to work on features independently of each other. To do this, I want to create a branch for each feature under development. However, I'd also like to do a continuous integration build of *all* features currently under development, so I can work out the kinks of putting them all together. But there's no automated way of doing that. With MergeMagician, I can configure the server to automatically merge the separate features into a single integration branch, where I can build and test everything together. There are lots of use case scenarios other than that one, but this use case is the basis of the MergeMagician concept.

I've gotten quite a bit of interest in MergeMagician so far. Even though the product is still under development, I want to stay in touch with these early prospects to keep them interested. To do this, I'm pioneering something I'll call "demo-driven development". It's very agile-esque, so it's not an entirely new concept. I'm sure someone else has thought of it and given it a different name. The idea is to write the minimum amount of code necessary to showcase an idea, feature, or concept of the product. When it's working, you shoot a narrated screencast of how it works, and then send it to your prospects. Because you're only sending out a screencast, rather than the real software, the feature doesn't have to be fully implemented. It just has to be good enough to demo it within a screencast. When that's done, you go onto the next thing, and repeat the process until the product is shippable.

The advantages of this approach are 1) I get feedback along the way, 2) Keep prospects interested in the product, and 3) Build product mindshare before it's shippable.

We'll see how effective this approach turns out to be. It seems to be working so far. The first screencast I did wasn't even the real product, it was just a wireframe mockup of the product. I sent this to a few people, and they in turn forwarded it on to some people. Next thing I knew, three people from Microsoft contacted me and wanted to know more about the product! I'm almost ready to shoot my next screencast, and this time, it will be real, running code.

Discovering stackoverflow.com

2009-04-29T10:49:09.558875-05:00

I was listening to Scott Hanselman's podcast #158 (http://www.hanselminutes.com/default.aspx?showID=176), which was an interview with Joel Spolsky, founder of Fog Creek Software.

A couple of take aways ... first, Joel Spolsky is a wickedly smart dude. BTW, this is the same "Joel" that I was picking on a few days ago in my "Kevin On Hiring" blog post. I still stand by the things I said in that post, but it makes me nervous when I start picking a fight with someone of Joel's intellect.

Second, stackoverflow.com is pretty cool. I've run across this site a few times while Googling for stuff, but now I'm starting to go directly to stackoverflow.com for searches on programming-related questions.

I've never been a huge fan of Google. I use it, of course, and I get useful results from it from time to time. But a lot of times, man is it frustrating. It isn't optimized for searching for programming-related questions. That's the problem stackoverflow.com was created to solve. It's community-oriented, so users can vote on the best answers to questions, and if you participate in answering questions, you can build up "reputation" points. Like I said, it's pretty cool. To learn more about the motiviation behind stackoverflow, listen to the podcast above. Then, give stackoverflow.com a whirl.

VSTS Database Edition

2009-04-22T22:43:31.812-05:00

Today I attended a Microsoft "Big Event" in Denver that covered Visual Studio Team System.

A lot the content was already familiar to me, but I did appreciate Rob Bagby's demo of Visual Studio Database Edition. I've known about it, but haven't used it, and wasn't sure if it was something that would help me or not.

There is some good stuff in there, such as database unit tests and nice tooling for generating sample data to test against. Overall, however, I don't think Database Edition is really geared towards the way I work. Let me explain.

For nearly all of my 20 year career, I've worked in small startup companies developing database oriented applications. I've developed two defect tracking systems (PVCS Tracker and TeamTrack), a build / continuous integration system (BuildBeat), and currently, an automated branching & merging system (MergeMagician). Although the development language changed over the years, all of them used back-end relational databases and some sort of ORM mapping layer. In all cases, the database was created by the user of the software (none of them were SaaS solutions), using an administrative component and/or wizard that was embedded in the system itself. Having created the database, the system would then populate it with data as the user used the system (submitted defects, ran builds, etc.)

In this mode of operation, information about the database schema, starts in the application, usually in the form of an XML metadata file that is associated with the ORM mapping layer. Using this metadata, the application dynamically generates SQL statements and issues them to the database. This seems a very natural way to work, and because it is so ingrained in how I've worked, it's hard for me to envision doing it any other way. It turns out, however, that in IT-oriented environments, that database in many cases pre-dates the application under development and already exists. Developer's in these cases need to transform or import the database schema into a form that programmers can work with. As the code is modified and deployed, you run into synchronization problems with keeping your code, your development database, and your production database aligned with each other. This is one of the main areas the Database Edition targets.

Using the database that already exists (once again, an exasperating concept to me), you import the schema with Database Edition and you get a bunch of .SQL file injected into your project. That means that "the truth about your schema" can now exist independent of the actual database instance. Since they are regular text files, you can check them into source code, version them, and do all the same sorts of things that you would do with ordinary source code. You can also now create new database instances, using the single schema truth (your .SQL files in your project). That makes perfect sense, and once again, the fact you would do anything other than 1) Create file(s) that describe the schema you want, and then 2) Create a database instance using that description, makes it difficult for me to see this concept as revolutionary. But apparently it is to a lot of people.

The problem for me, however, is that I never work directly with .SQL files. I always work with a more descriptive file format (usually XML) that not only holds the information that is needed by the database to initialize the schema, but I also store extra metadata in the file that assists in other areas, such as generating documentation and generating code. The descriptive XML file works in conjunction with the ORM layer and/or code generator so that you get what you want. Microsoft recognizes this way of working with two technologies, LINQ to SQL, which uses a .DBML file format, and Entity Framework, which uses .EDMX, both of which are XML based. I have my own ORM mapping layer, which also uses an XML-based metadata file. The SQL statements are either dynamically created at runtime, or they are code generated. I think most ORM tools on the market work this way. It insulates you from working directly with .SQL files. This is why I have a hard time seeing how Database Edition would be all that useful to me.

I tried to explain these thoughts to Rob Bagby, who, not surprisingly, didn't agree with me. He asserted that storing the "schema truth" in .SQL files was the right way to do things because "SQL is the industry standard." For me and my way of working, I still don't buy it. SQL is a bad file format for me for the following reasons:

* It's hard to parse, and thus hard to manipulate with external tools. With an XML file format, I can create an XSD and then code generate a data binding layer to read in the file and process it however I want. With SQL, I have to break into heavy duty lex/yacc and start dealing with language grammars. I'm sure there are a lot of SQL parsers out there I could use, but it's going to be more work for me.
* It doesn't store enough information. I want something that not only stores information needed to create the schema, but extra metadata, such as prefixes, name transformations, namespaces, etc., that will feed into my code generator.
* It's not extensible.
* If I target multiple databases (such a MySQL and Oracle), using SQL is dangerous because different databases use slightly different syntaxes. With an XML file format, I can make different transformations to handle other database types.

For me, SQL is an intermediate file format, not an input file format. People who work the way I do, are more in need of a tool that will synchronize their XML metadata file (.DBML, .EDMX, or other) with the database schema. I know a lot of people that are struggling with this very issue.

I've started on an open source project to do this sort of thing. It's called DbmlManager, and it's available at http://www.codeplex.com/dbmlmanager. It's not very far along however, but if anyone wants to work on it with me, let me know.

Kevin On Hiring

2009-04-16T00:13:00.7335-05:00

I was at the bookstore today and I was flipping through Joel On Software when I came across the chapter on hiring. As is typical of other business books, it promoted a "hire the best" philosophy. I've pondered this notion over the years, and wanted to share my thoughts.

Let me first say that I agree with the underlying philosophy of the advice, that being that a small number of highly-motivated, talented people can outperform an army of mediocre individuals. However, there are several aspects of the "hire the best" mantra that have always bothered me.

First, it has unfortunately become a tired old management cliche. Of course everybody says they hire the best. Who would ever boast about hiring the worst? And who would ever want to work for a company that brags about hiring the worst? But saying you hire the best, and actually hiring the best are two very different things.

Second, it's a bit absolutist. In my mind, if there are 17 million programmers in the world, then "hire the best" means 1 person has a job and the other 16,999,999 people are unemployed. That's obviously impractical, making the advice at the most general level non-actionable. I hate non-actionable advice.

Third, there really is no such thing as "the best". There are a variety of factors that make up a good associate - raw intelligence, communication skills, people skills, work ethic, etc. Every potential employee will have a unique mix of these aspects ... no one is perfect at everything.

Fourth, what does it say about me? Am I the best? I do the very best I can to "sharpen the saw" of my personal skill set. I read programming books and magazines, self-teach myself new technologies, read blogs, and attend all manner of workshops, user groups, and code camps. I feel like I'm at the top of my game, and the best programmer that I've ever been. But am I the absolute best? In a world filled with so much talent, it would be very conceited of me to make such a claim.

Lastly, what do you do if you live in a small city like I do that isn't the epicenter of the technological world? Should I move to San Jose, CA or Redmond, WA, where the per-capita talent may be higher than where I live? And how can I compete as a small software company if all the other larger software companies have already gobbled up the best talent?

So here's my take on it. I don't think you need to move to California or Washington. I think there are plenty of smart people all around ... you just have to find them. I think you should hiring the smartest people you can find, invest in them to help them be the best they can be, and encourage them to fanatically acquire domain knowledge.

The latter two points above seem to be glossed over too quickly by "hire the best" aficionados. Yes, raw talent is a very important aspect, but I think great programmers are born over time, not stamped out at a factory. And I want people who care intensely about what they're doing and want to learn everything they possibly can about their domain. There are just too many people in the world that want to punch a clock and pick up a paycheck. Give me people with passion.

Which brings me to my closing thought ... leadership. You can have the best people in the world, but without great leadership they won't accomplish much. And in a small software company, leadership pretty much comes down to one thing ... YOU.

Using Visual Studio 2008 T4 Code Generator

2009-03-27T11:41:50.98-05:00

I've been using T4 to do some code generation in Visual Studio 2008. I shot a 4-part screencast series that describes the entire process.

In the videos, I talk about two open source projects, both available on CodePlex.

The T4 Toolbox is available at www.codeplex.com/t4toolbox.
XmlGen# is available at www.codeplex.com/xmlgensharp.

Videos
Part 1 - Creating XML Data Bindings using XmlGen#
Part 2 - The XML data file and T4 code generation harness
Part 3 - The T4 template
Part 4 - Using TemplateEx to manage whitespace

Secrets of Success - Part 3

2009-03-25T15:11:48.293-05:00

Back in the early days of my career, it seems you could just pick an idea, do it, and then sell it. Times are different now. How do you navigate the entrepreneurial waters in this era?

Getting back to my first blog entry in this series, I certainly agree with my sales and marketing friends that disconnecting from the world for a long period of time while attempting to write a perfect product is dangerous. On the other hand, I've had a lot of challenges trying to utilize the personalized, one-on-one interview technique that they suggest. Some challenges I run into include:

1. It's a process that doesn't fit well with my core personality.

2. Identifying good people to interview is hard. For me, interviewing software developers doesn't seem to be good enough. I need people with expertise in ALM to get valuable feedback about the ideas I have.

3. Despite the assurances of my sales and marketing friends, no common pattern of need has emerged as I've done these interviews over the past few years.

4. It's an emotional rollercoaster. If I talk to someone that gets what I'm talking about, I'm excited and ready to start cranking out code. If they don't get it, I get discouraged and wonder if it's really a very good idea or not. I suspect that I might have thrown in the towel on some good ideas because the handful of people I talked to about it didn't get it.

5. You have to conduct a large number of interviews to get any statistically valid data.

6. Most people don't give you actionable feedback. Most people just say something very generic like "That sounds interesting."

So what do you do? I have a process in mind that I think, based on observations of other successful startup companies, will work. To describe the process, I'll start with an analogy.

Suppose you're an adventure tour operator specializing in shark petting. You want your tourists to see as many sharks as possible. So how do you go about bringing the sharks and the tourists together?

In one approach, you start with a detailed map of the ocean floor. You conduct detailed data mining on every shark that's ever been caught by a fisherman. You then dive down into the ocean with a remote controlled submarine to make contact with each individual shark. When you find a shark, you tag them with a tracking device. Then, using the world's most advanced radar system, you map the movement of each shark, feeding all of that data into another data mining program. From here, you discover where the highest concentrations of sharks live. You take your tourists to that location, dispatch each tourist in their very own submarine, and chase the shark down to get a glimpse of it.

Pretty complicated approach, right? It's hard to do because the ocean is so big, and the sharks are too few. Let's look at a simpler approach that's more doable.

Armed with some general knowledge of shark migratory patterns, you drop anchor in a location where you think you have a reasonable chance of attracting some sharks. You then drop chum in the water to get the sharks to come to you.

And that's how I think you have to do things. The location represents your product idea. The sharks represent your potential customers. The chum represents activities that you do that will get potential customers to come to your web site and take a look.

The key is to write code and chum at the same time. If you only write code, then when the product is released you'll be dropping it into a world that is unprepared for it. One of the main reasons for chumming isn't just that it provides research data, but that it builds buzz and momentum so that the market is primed and ready for the product.

How do you chum? Try these things ...

· Get the product web site up and running with software that supports groups and forums, so you can communicate with visitors.

· Start an early access program. Some information should be available to everyone, but detailed product information and downloads should require registration.

· Contact everyone on your Facebook friends list and your LinkedIn network to let them know about the product web site and early access program. Invite people to join the early access program.

· Write white papers and make them available on your own web site and other applicable web sites, if possible.

· Start your search engine optimization efforts prior to product launch.

· Start your own blog. Don't just blog directly about the product, but also blog about tangentially-related topics.

· Visit other people's blogs and post comments that link back to your web site.

· Get other people to blog about the product.

· For people registered with the early access program, publish early mockups and screencasts about the product. I'm using Balsamiq Mockups for this (www.balsamiq.com). Create a forum on the web site for people to provide feedback.

· Start an open source project that provides functionality that is tangentially-related to the product you're working on. If such open source projects already exist, become a contributor to those projects.

· Join mailing lists or Internet groups that serve users with similar needs or desires to your product. If none exist, create your own.

· Use agile software development practices to get incremental beta releases in the hands of beta testers early. Try to get something useful out fast.

All of these things are great, after you've settled on a product idea you want to pursue. But that still leads back to the original question I posed, how do you decide upon the idea itself? My advice is stop looking for a perfect idea. It doesn't exist. Pick a problem area that you've personally experienced. If it's a problem for you, it's a problem for other people also. Pick something you're passionate about. Pick something unique. Create. Innovate. If the product space you're going into already exists, then try to figure out at least two or three things that will make your product unique from all others.

Then, just do it.