In time larger companies will start to move back towards the lower cost of free MVT tools in favour of diverting the budget towards in house analysts or consultants to make better use of the results. In the same way as is happening with Google Analytics.

Brian – What you really should have done was commence testing BEFORE you made any permanent changes to your website. Unless the site was severely outdated and there were several major issues, you should have run a series of tests on different elements of the site and fed them into your new site plan. The core purpose of testing is to measure the changes so making major unmeasured changes to your site before starting a series of smaller measured changes might not be the best way to go in the future.

If you are considering a major re-design in the future you should also employ some user input through usability testing or focus groups, again to input into the changes you intend to make BEFORE you make them. They prove very valuable to site re-designs.

Hope that might be useful,
Matt

Not a bad looking strategy too! my only bugbear with how writing strategies is taught is that it always seems weird to set ‘SMART’ objectives without understanding the potential costs to achieve them. An example is “increasing organic traffic to 10,000 per month in 6 months” then tootling over to the keyword research piece only to find that to rank for the keywords that will fulfill this objective means throwing 1,000,000 links at the site at a huge cost thus delivering a massive CPA and poor ROI. I tend to look at the budget before setting any ‘Specifics’. How about you?

From a UX / usability consultant’s perspective this is an interesting post and comments. Agree completely over people being more important than the technology you use.

It would be good to hear a Maxymiser response. Also I wondered if you should be clear in the intro why you didn’t get a return from Maxymiser – it currently reads as a negative review! (I have no affiliation!)

Cheers
s

I agree with most of your points, especially your last point. I’ve learned now that I’d rather have a qualified consultant and a free tool, than an expensive tool with no real ‘human’ insight behind it. I guess it’s the same logic that Avinash Kaushik uses when he states that only 10% of an analytics budget should be used on technology. The rest should be spent on getting quality people onboard to advise on what that data means for the business and how to leverage it for business results.

On your first point, I’d like to apologise if my post wasn’t clear in this area. We did not start MV testing on our old site. Maxymiser deserve a thumbs up here for advising us to only start the testing process once our new designs were live, that way would could use all of our ‘learnings’ to see a benefit straight away. So while we were working with Maxymiser before our sites were redesigned, this was purely on a consultancy basis. They did not start multivariate testing until our new site was live.