Email Updates RSS Subscribe
Line

This blog is created and maintained by the technical team at Hook in an effort to preserve and share the insights and experience gained during the research and testing phases of our development process. Often, much of this information is lost or hidden once a project is completed. These articles aim to revisit, expand and/or review the concepts that seem worth exploring further. The site also serves as a platform for releasing tools developed internally to help streamline ad development.

Launch
Line

Hook is a digital production company that develops interactive content for industry leading agencies and their brands. For more information visit www.byhook.com.

Line

WTFFFFF: Online Flash Quality Assurance Platform

Line
Posted on June 30th, 2010 by Jake
Line

WTFFFFF! We all know the feeling. You just sent off a build to the client about fifteen minutes ago, and are finally packing up for the night. Then the phone rings… deep down you hope its good news, and its the client praising you for all of your hard work, but come on, you know better. It is of course the client, calling to tell you that the build works great for him, but he sent the link to his friends and family to show off what “he” made, only to get an email from this third cousin’s friends daughter that “something” doesn’t work right. *Sigh*

So you run through the first steps of trouble shooting:
1) What error are they getting?
Answer: “She didn’t say”

2) What kind of computer are they having the trouble on?
Answer: “I don’t know”

3) Which version of Flash player are they using?
Answer: “I couldn’t say…”

4) Can you send this questions to her, so we can troubleshoot?
Answer: “No, she won’t be home tonight…”

I’ll spare you the pain of all of the other questions I’m sure we all would ask.

As usual the real issue here is not being able to communicate with the people having the trouble. Worse yet, the people doing the testing aren’t really QA people, they are just random folks. So we decided to try to come up with some tools to help with this:

The WTFFFFF Online Flash Quality Assurance Platform

The platform is made up of three distinct parts:
- The Developer Landing Page, where developers set up the tests and generate a link to send to the client that is having issues.

- The Client Landing Page, which is the first thing the client sees after clicking on a generated link created by the developer main page.

- The Modules Pop-Up which does the actual data collection and email submission.

The Developer Landing Page
WTFFFFF Developer Landing Page
This page can be access from http://www.wtfffff.com and is used by a developer or QA person that wants to get more specific data about an issue a client is having. This is accomplished by simply filling out the form, checking which modules to run, and clicking “Generate QA Link”.

The Send Reports To field is where you can put a comma separated list of email address to where the results from the test will be sent. This will most likely be the email address of the person generating the link.

The URL To Test field is the link to the page containing the swfs that are to be tested. If you are creating an ad inside of an Ad Kit, you would put the ad kit’s preview link there. As another example, if you were creating a full flash site or even a small flash piece that fits into a traditional site, you would put the url to the full site there. As you will see in a minute, this link is opened along with the Modules Pop-Up, so data can be collected from the swf, and sent back through the Modules Pop-Up.

The check boxes further down the page are used to select which test modules you want to load in the Modules Pop-Up. More detail will be provided later in this post regarding each of the modules’ functionality.

The last step in the setup process is to generate a link that a developer or QA person can send to the client. When “Generate QA Link” is clicked, a full url is generated that includes all of the options that were set in the form. This link can now be copied and emailed to the person that is having the troubles. All they have to do is click on the link, and when the tests are done, click the “Submit Report” button in the Modules Pop-Up.

For convenience there is a “Go To Link” button which will let the developer test the link before sending it out to the client. After clicking that button, the developer/QA person is taken through the same process as the client after they click on the generated link.

The Client Landing Page
WTFFFFF Client Landing Page
Once the QA Link is generated and sent off to the client (via email, im, or whatever), the client can then click on that link which will take them to the Client Landing Page. This page lets the client know what to expect before they click the “Launch Diagnostic” button. Once they click that button, the Module Pop-Up is displayed, and all of the selected modules are loaded. After the modules load, they will execute their initial tests, one at a time. Once those tests are complete, the Client Landing Page is redirected to the url that was in the “URL To Test” field on the Developer Landing Page. At this point the modules will do their thing while the page to be tested is running in another window.

The Modules Pop-Up
WTFFFFF Modules Pop Up
Currently there are four modules that we have created (with more to come). The Data Collector, Log Collector, 3D Performance Test, and Debug Player Detector.

The Data Collector will gather a bunch of information about the machine that is running the tests, such as browser version, flash version, cpu architecture, os version, and a bunch of other platform related bits of information.

The Log Collector gathers all of the log statements produced from the swf being tested. To make use of this however, you will need to use our Logging Library, and the LCTarget log target. You can grab the library here:
http://labs.byhook.com/2010/04/05/logs-theyre-better-than-bad-theyre-good/

Once you have the library, you need to add the LCTarget like this:

Log.addLogTarget(new LCTarget());

Then to log something:

Log.log("This is a sweet log entry");

Now anytime anything goes through the logging system it will be collected by the Log Collector module.

The 3D Performance test is the standard Away3D performance test. The test will add 3D objects to the display until your framerate reaches 15 frames per second. Once the target framerate is reached, the performance info is collected and stored in the report that will get sent back to the Developer/QA person.

Lastly, the Debug Player Detector will check to see if the Flash Debug Player is installed. If it is not, the module will prompt the user to install it.

The other feature of the Module Pop-Up is the “Feedback Notes” field. You may want to encourage your client to fill out their name and any steps that they took to produce the issue.

When they are all done, they can simply click “Submit Report”. Once that happens, an email will be sent, with all of the collected data, to the Developer/QA Person.

That’s the nickel tour of WTFFFFF. As always, if you run into any issues, or would like to see specific modules added, please let us know in the comments below. We would love to make this as useful as possible to help ease some of the pain associated with debugging on seemingly endless platform combinations.

Line
4 Responses to “WTFFFFF: Online Flash Quality Assurance Platform”
  1. Tahir Ahmed says:

    Wow guys, that is good. With so much info coming from the user’s end, it will definitely help in the debug process.

    Kudos,
    Tahir.

  2. jake says:

    @yanekx: We were probably a little loose with the wording on that. In this case its more CPU than GPU. The rendering is all done on the CPU, but some of the drawing has been optimized in FP10.1 to utilize more of the GPU if possible. So, really CPU stressing.

  3. yanekx says:

    hi. this is fine tool. could you explain what you mean by “Runs a GPU/CPU stress test with Away3D.”? Especially “GPU” part… How GPU stress test is done?

  4. [...] This post was mentioned on Twitter by Ultra Red. Ultra Red said: WTFFFFF: Online Flash Quality Assurance Platform: WTFFFFF! We all know the feeling. You just sent off a build to t… http://bit.ly/aSt5ND [...]


Leave a Reply

*

Line
Line
Pony