What is most valuable?
There are several features that are very valuable to me, and these are the main reasons I was attracted to it.
- It has a very wide array of built in functionality. I am able to test against SNMP (v1 and v2), Telnet, SSH, Command Line, Web Browsers (IE, Google, FireFox), email, Microsoft Office, Windows integration, etc… On top of all that I can write other apps in other languages and have Testing Anywhere control them when necessary.
- The scripting language is easy to learn and very quick to build on.
- The server side allows for multiple developers to be involved and all their work is saved to a single repository.
How has it helped my organization?
We chose this product because we had an extremely aggressive timetable to develop automation from scratch. In a traditional setting it would have taken several developers months to develop a full of suite of tests for just one feature set. Using Testing Anywhere we were able to leverage the built in capabilities such as Telnet and product a quick turnaround in developing a Telnet menu test that took a matter of weeks to develop.
What needs improvement?
There are a few areas that this product really needs to bring to the next level. First, the server is good but very basic. It does a great job of storing all the test files in a central location for all developers to use. However, it lacks all the standard features a normal “code” repository has such as Check In/Check Out, Branching, Version Control, etc…
Second, the IDE – or development interface is simple and easy to use. But if you come from a coding background such as myself you’ll find that it is a little too simplistic. I've become accustomed to the Microsoft Visual Studio IDE where it is very functional and helpful in developing your code. The Testing Anywhere IDE lacks all the bells and whistles you get in a full features IDE such as Visual Studios.
For how long have I used the solution?
I have been using this solution for about 1 ½ - 2 years now.
What was my experience with deployment of the solution?
Testing Anywhere offers two licenses for deployment – a runtime license and an EXE license.
The EXE license is such that I can create “executable” files from the developer machine and deploy it. In creating the EXE you include all of your scripts necessary for it to run. However, there is no control on how or when those are created and the file that you created can only be deployed to one system. SO if you wanted to deploy this test file to say 10 test systems you would need 10 licenses.
The runtime license is basically the same as a developers license without the ability to edit. I've come to believe this is very useless as I have to manually put in all my scripts and manually check them every time I run (because there is no way to verify what is running is the latest). One of our tests involved 200 scripts, so this can be a rather tiresome venture. To Testing Anywheres' credit, there is supposedly a different server than I am using, and that allows you to update the scripts on a remote machine but we are not using that nor have I seen it so I cannot comment on its abilities.
What do I think about the stability of the solution?
We have encountered a few. My company was one of the early users (if not the earliest) of the Testing Anywhere product. I'm told we used the product in ways that was not anticipated and pushed the boundaries of the product and development teams at Testing Anywhere. Because of this we uncovered a number of bugs that we were able to work with the developers at Testing Anywhere to get resolved.
What do I think about the scalability of the solution?
We have run into a few. First, the server we purchased is limited to just a handful of licenses - six I believe. This includes developers and runtimes. This may seem like a lot but if you include all your test machines (the run times) that quickly dissipates. We are currently discussing evaluating the next level of server offered called Test Lab which is supposed to resolve this issue.
How are customer service and technical support?
I would rate them as 5/10, and only because I have had most of my interaction with technical support and our product manager. I have not had to contact customer support much at all – Which is a good thing! Technical Support
I would rate them as 8/10. They are always responsive. I am often in contact with someone on their team ensuring that they can re-produce the issues and working with them to find a solution.
Which solution did I use previously and why did I switch?
I come from a background where we developed all of our test tools internally. I’ve used mainly Visual Studio. The only reason I entertained and then chose this solution was because of the time table and resources made available by my management.
How was the initial setup?
I found the initial setup rather easy and straight forward. I think coming from a programming background made it that much easier but others that I have worked with that did not have a programming background have not had much difficulty either.
What about the implementation team?
This was done through an in-house team.
Which other solutions did I evaluate?
Yes, we did look into several other options. We looked at an internal team of developers developing in some type programming language. We also looked at Selenium, Squish, and a few others.
What other advice do I have?
If you are looking for a fast solution using minimal resources not requiring an advanced programming skill set this is your product. It’s easy to learn and quick to deploy.
While it has some really great features and is extremely easy to use it is still fairly new to the market and has its bugs. (Not that other solutions do not).