Enlighten was acquired by Perficient Digital in December 2015
When Enlighte (now Perficient Digital) embarked on a redesign project for Jimmy John’s Gourmet Sandwiches, it was a little bit daunting. It would be a massive undertaking, with Enlighten leading other vendors in combining multiple sites and platforms into a unified user experience for a brand with a loyal following and a reputation built on being “freaky fast”. This was a job for responsive web design (RWD), an approach that results in a single web app optimized for multiple device sizes and types.
Our quality assurance (QA) team had been testing mobile websites for years, but RWD on this scale was something new. Given the complexity of the project—one that included not only the main www.jimmyjohns.com site, but also online ordering, e-commerce (for merchandise, gift cards, etc.), and the benefits of QA working closely with engineering—we decided to keep the testing efforts in-house rather than outsourcing them. Here are some of the lessons we learned:
1) Stock your QA lab with a variety of dedicated devices; don’t just rely on what you’ve got in your pockets. With many employees carrying their own smartphones, smaller companies may be tempted to adopt a BYOD (bring your own device) approach to testing. But if a tester logged a bug last week, using a phone that belongs to a coworker who’s on vacation this week, how can you be sure to reproduce the issue or validate the fix? For consistent testing you should always have an assortment of dedicated phones and tablets on hand.
Which ones you use will depend on the size of your testing team, your budget, and your target audience. Apple’s iPhone and iPad should be part of any testing setup, and you’ll want to cover Android phones and tablets as well. The fragmented state of Android—split among multiple OS versions with add-ons and customizations from manufacturers and carriers—means it’s harder to choose which devices to test, but you’ll probably want to lean towards the most popular ones (like the Samsung Galaxy or Google Nexus families). Beyond that, there are Windows devices—phones, tablets, and even touchscreen PCs—as well as Kindle Fire tablets and Blackberry phones. Look for statistics at sites like StatCounter, NetMarketShare, or Android’s Developer Dashboards to get a rough idea of what the market looks like.
If money’s tight, consider the iPod touch instead of the iPhone, or a low-end, no-contract Android phone instead of the latest model. Mobile testing can also be done on the desktop using emulators, either locally or via a remote service like BrowserStack. Just be aware that these emulators may not behave 100% identically to the real devices, and don’t entirely replicate the experience of using an actual phone.
Lastly, a little redundancy in the device inventory might not hurt. If you have to share devices, keep a sign-out sheet so you always know who’s got what you’re looking for. And try to keep some extra charging cables around, just in case.
2) Decide on what devices, OSes, and browsers to support on each project ahead of time…but not too far ahead. With most projects, we come up with a Browser Matrix before development starts, specifying exactly what we’ll be using to test the site. With Jimmy John’s, the development cycle was long enough that the matrix was already a little dated when it was time to start testing. We made adjustments to keep up, but had to be careful not to overcommit to supporting things the developers didn’t have in mind when coding. It’s best to come up with guidelines at the outset and revisit them occasionally during the life of the project to see if any tweaks are needed.
Don’t forget that mobile devices can run multiple browsers just like desktop computers do. On Android 4.x, the default may be the stock Android browser or it could be Chrome, depending on the OS version and manufacturer, so we ended up testing both. Other browsers like Firefox, Opera, and UC Browser are available for download—and are very popular with users in some countries.
But remember that support for a particular device, OS, or browser doesn’t have to be all-or-nothing. For older devices, or ones with smaller market share, you may decide to spot-check key functionality and not worry about visual quirks.
3) Test responsive page templates or prototypes before the code is fully integrated. For some parts of the project this was unavoidable, especially in cases where we were handing off HTML templates to other vendors to integrate their backend code. But even in the areas that we were developing completely internally, it was very helpful to find and debug layout issues on page templates during an early testing pass. We could then concentrate more on functional testing during the full QA phase. The original responsive (yet nonfunctional) templates were kept available too, so we could refer back to them in lieu of static design comps and compare the finished product to the original designs.
4) Think outside the desktop. If you’re used to testing sites in desktop web browsers, touchscreens bring their own set of concerns that may be new to you. Are links and buttons large enough for your fingers to tap easily without hitting the wrong one? What happens if you load the page in horizontal orientation and then turn the tablet to vertical, and back again, or vice versa? Does your phone’s browser mistake an order ID for a phone number and try to format it like one? Does the on-screen keyboard default to numeric mode when using a numeric form field? Mobile testing means learning some new tricks and breaking some old habits.
5) Allow more time than you think you need. No matter how light and fast your responsive or mobile-optimized site is, the experience of testing a site on a phone tends to take longer than using a desktop browser—especially if the Wi-Fi network is acting up that day! It’s not always safe to assume a task on a mobile device can be knocked out quickly, or that mobile testing is only a minor addition to your project estimate.
Our experience with RWD meant a lot of hard work and more than a few late nights. The end result: an award-winning site that looks great whether you’re browsing it from a big-screen monitor, a 3.5” smartphone, or something in between.
