As Bojan Simic, president of IT analyst firm Trac Research, points out in new research note, “application performance management” (APM) has generated a ton of buzz over the last couple of years. Numerous vendors, be they providing APM specifically or wrapping APM functions into traditional management platforms, have glommed onto the term, promising value propositions such as end-to-end visibility and improved IT-business alignment.

“APM has gotten to the point where the term doesn’t mean a whole lot anymore – when you see demos and talk to customers, it’s obvious that many ‘APM’ vendors really shouldn’t be competing against each other. People have got to understand that the APM bucket really consists of five or six different classes of technologies,” Simic says.

What that means, he adds, is that before enterprise IT managers explore their APM options, they must carefully assess their application environments – how many users are using what sorts of applications, for example – and then determine what exactly they want from an APM tool. Do they want to monitor the end-user experience or business transactions, for example? Do they want to feed APM data into and integrate with business service, network performance or database monitoring platforms? Coming to that sort of understanding will help narrow the field of choices, Simic says.

Meantime, he adds, IT managers can expect to see tool vendors focus in on proactive management capabilities in 2011.

“Today when APM vendors say something is being proactively managed, that oftentimes means the tool sends an alert and then you try to figure out what’s going on. But to be really proactive, you need to understand the problem is coming before people start complaining about it – and vendors are working to differentiate themselves on this now,” Simic says.

In addition, vendors also will be working on enabling deeper views into the user experience, he says.

“It’s becoming more important to do more than actually just measure availability or speed of an application,” Simic says. “You want to know if an application is being used and, if so, which particular features. You also need to know about business impact – if an application performance problem is only affected two vs. 2,000 people, fixing the problem might not be so pressing.”

Lastly, he adds, in 2011 APM vendors also will distinguish themselves on how well their tools do at tracking the performance of applications as they move dynamically around a virtual environment.

NetworkWorld

(Original Article)

“Know your users” has long been a guiding principle for IT managers assessing and selecting new technologies. But when dealing with application performance issues, IT’s motto needs to be “know what your users are experiencing.”

That’s because if you monitor how real users interact with applications as part of their daily work, you’ll receive fewer complaints about application issues — and who doesn’t welcome fewer resource-grabbing, time-consuming help desk calls?

Help desks prep for consumer device blitz

In a newly published report, Aberdeen Group noted that the need to curb user complaints about application responsiveness and usability top the list of reasons companies adopt user experience monitoring software.

For the report, Aberdeen this summer conducted a survey of 170 organizations about end-user experience and monitoring. Investing in this type of monitoring software is proving a good value for enterprises surveyed. The results “provide solid evidence that companies that monitor the end-user experience have fewer end-user complaints with application issues and the ability to use this experience to build applications that are easier to use and require less resources to support,” said Jeffrey Hill, a research analyst with Aberdeen’s Technology Practice, in the report.

At best-in-class companies, Aberdeen found, IT uncovered 53% of application issues through end-user experiencing monitoring while realizing a 48% improvement in the mean time to repair application performance issues discovered through end-user experience monitoring. In addition, best-in-class companies gained a 42% increase in visibility into critical business transactions and a 15% decrease in the number of end-user complaints about application performance, the report noted.

Aberdeen also found several common characteristics among best-in-class companies. For example, 65% use the user experience as part of the application design or when making changes, 63% have the ability to associate user transactions with the business, and 50% monitor the user experience across multiple operating systems, the report cited.

“The message is clear,” Hill said in the report, “industry average and laggard companies need to make measuring both application and end-user experience a priority.”

Read more about infrastructure management in Network World’s Infrastructure Management section.

Schultz is a longtime IT journalist. You can email her or find her here.

Absolute Performance offers unique end-user experience monitoring through its WebWalk product line.  Click to learn more about why end-user experience monitoring is worth while.

Retailers Integrate End-User Experience to Increase Application Usability and Reduce User Complaints

BOSTON, MA, Sep 08, 2010 (MARKETWIRE via COMTEX) — By integrating the experience of end users with application design, companies in Aberdeen’s study have improved application performance, reduced complaints from end users about application issues, and have noticed an overall increase in the quality of the end user’s experience. The latest research report, End-User Experience Monitoring and Management, announced by Aberdeen Group, a Harte-Hanks Company /quotes/comstock/13*!hhs/quotes/nls/hhs (HHS 10.74, +0.09, +0.85%) , finds that companies that implement monitoring of the user’s experience also see improvements in application availability and response times, increased visibility into the performance of business transactions, and a reduction in the average time it takes to find and repair application issues.

The research report delivers hard-hitting facts based on responses from over 170 companies, and investigates the steps that top performing companies have taken to use the experience of end users to improve their business.

The report reveals the strategies top performing companies use to meet their application performance management requirements. Aberdeen’s research found:

--  Sixty-five percent (65%) of top performing respondents use the
    experience of end users as part of application design or changes
--  Forty-seven percent (47%) of companies that are successful in
    monitoring the experience of end users had continuous visibility into
    the performance of revenue-generating transactions, as compared to 23%
    of all other

“Today, companies want to move beyond traditional measurements of application performance,” says Jeff Hill, Research Analyst and author of the study. “Companies are looking for a solution provider with a demonstrated success in implementing solutions, superior customer service and training, and deep domain expertise that offer the ability to integrate the experience of users into business-critical applications and processes.”

You can access a complementary copy of the End-User Experience Monitoring and Management report from Aberdeen Group’s website.

For more information on this report, please contact kevin.permenter@aberdeen.com or jeff.hill@aberdeen.com.

A complimentary copy of this End-User Experience research report is made available due in part to the following underwriters: Knoa Software and Tevron.

Visit Research.Aberdeen.com for additional access to complimentary Information Technology Research.

About Aberdeen Group, a Harte-Hanks Company Aberdeen provides fact-based research and market intelligence that delivers demonstrable results. Having studied more than 30,000 companies in the past two years, Aberdeen educates users to action by driving market awareness, creating demand, enabling sales, and delivering meaningful return-on-investment analysis. As the trusted advisor to the global technology markets, corporations turn to Aberdeen(TM) for insights that drive decisions.

As a Harte-Hanks Company, Aberdeen places content in context for the global direct and targeted marketing company. Aberdeen’s analytical and independent view of Harte-Hanks’ “customer optimization” process (Information – Opportunity – Insight – Engagement – Interaction) extends the client value and accentuates the strategic role Harte-Hanks brings to the market. For additional information, visit Aberdeen http://www.aberdeen.com or call (617) 723-7890, or to learn more about Harte-Hanks, call (800) 456-9748 or go to http://www.harte-hanks.com.

To learn more about End User Experience Monitoring and Management visit the Absolute Performance website.

Copyright 2010 Aberdeen Group, Inc., a Harte-Hanks Company 451 D Street, Suite 710 Boston, Massachusetts 02210-1928 Telephone: (617) 854-5200 Fax: (617) 723-7897 www.aberdeen.com

SOURCE: Aberdeen Group

Copyright 2010 Marketwire, Inc., All rights reserved.

Application management–particularly solutions with built-in automation designed to help organizations monitor and manage performance and availability from a cohesive, end-to-end view–are becoming top concerns for both IT and business executives, according to a new study conducted by research and consulting firm Enterprise Management Associates (EMA). In fact, in 53 percent of the companies EMA surveyed, C-level IT executives, directors and managers are shaping application management their organization’s application strategy, and in 42 percent of those surveyed, C-level executives are directly involved in planning, implementing or promoting application management, compared to 37 percent in 2008.  Director-level executives are involved in 29 percent of companies surveyed, versus 18 percent  in 2008. Moreover, application management acquisitions are increasingly being funded out of IT executive versus IT operations budgets, the survey found.

EMA conducted the survey in late 2009, querying more than 150 respondents that met certain criteria, including familiarity with enterprise application deployment, maintenance, monitoring or management as IT specialists, managers responsible for application support teams, or line-of-business professionals involved in application management. Potential respondents were also screened regarding the types of “enterprise applications” they were managing. Nearly half the respondents were from enterprise-sized companies, those with 10,000 or more employees. The remaining were fairly evenly distributed among smaller companies, varying in size from less than 250 up to 10,000.

Julie Craig, EMA’s research director for application management, says the increased involvement of executives reflects the realization that monitoring and managing today’s highly distributed and increasingly complex applications is getting tougher, and application performance and quality is being affected. “I’ve started researching end-to-end application management about three years ago. IT organizations are still, by and large, good at monitoring and managing infrastructure. Most are not that good at managing applications,” Craig says. “And these executives are the ones talking to customers and partners and hearing if things are working, such as if a portal is down and that’s affecting a line of business’s ability to bring in revenue.”

To address the more complex applications–some of which run in service-oriented architectures (SOAs) or on virtual servers, making them more loosely connected and more dynamic–organizations need tools that offer new levels of automation to discover and monitor all the systems, networks and devices. Simply adding more staff is no longer a solution. As Craig puts it, “the requirements are really beyond human capability,” Instead organizations are seeking automated management solutions that improve application quality and boost efficiencies. “IT has to move to higher levels of automation because otherwise they want to be able to afford to manage their applications,” Craig adds.

Not surprisingly, user calls are still the primary way that IT finds out about application-related problems. Thirty-two percent of those queried said user calls are the most common way they find out about application-related problems, down from 54 percent in 2008. Automation is helping: 25 percent said detection by monitoring centers is the primary way they learn about problems, while12 percent picked notification from application management products. Another 12 percent picked notification from domain-specific management products (i.e., network, database, etc.).

The top three application management pain points driving management investments for 2010 are intermittent problems with no obvious cause/solution (27 percent), troubleshooting/root cause analysis takes too long (19 percent), and changes not well documented or controlled (15 percent). 

Organizations’ top three product types they believe to be most beneficial include a so-called “Single Pane of Glass” that provides correlation, analytics and a holistic view of end-to-end application management, application availability monitoring/management and application performance management, the study found. “A single pane of glass that wraps everything together, this is the holy grail and is the number one request I most hear,” Craig says.

She’s quick to point out, however, that while application management tools have gotten more sophisticated and are starting to address many of the concerns, IT organizations need to understand that technology alone isn’t enough. “Application management isn’t just a technology problem, it is a business problem. Automation alone can’t do it,” she says. Organizations need to form Centers of Excellence and/or cross-functional groups to oversee application management that combine the IT operational knowledge and skills, such as network management and database administration, with transaction, application and business knowledge and skills.

​

Previously, we looked at laying the foundation for an APM solution, but now it’s time to get to work. We introduced Jim – a veteran IT manager who lacks any APM solution. As we discussed, he needs to establish key performance indicators around a critical application, including performance metrics and SLAs. His billing system also has a service catalog and detailed security and reporting modules built into the system. As a Web-based application, the backend database is distributed and the application serves just under 2,000 users. As Jim lacks application fault or performance monitoring, he really is starting from scratch. So we are going to start with creating a document Jim can use for his requirements and build his business case.

Jim’s first inclination was to search the Internet and analysts like Gartner for potential APM vendor products that might fit in his environment. After some discussion, we encouraged Jim to first build a business case that includes his requirements. Even though time is short, and Jim is under pressure to deploy a solution, to effectively evaluate the APM vendors he needs to understand what specific technical and business requirements he has. Otherwise, even the seasoned IT manager may get lost in a sea of marketing.

We worked with Jim to develop a template that breaks down columns for stakeholders, a description of the APM requirement, its priority, and the source of the requirement -if it came from a document or a person, including granular details citing the date and pages when appropriate. We helped break these requirements down into three technical categories: data collection, data aggregation/analysis and data presentation. Each of these areas were critical to understand the complete APM offering.

Jim wanted to identify the cause of the performance slowdown, not just that a problem exists. Data collection requires a detailed understanding of the application architecture in place, which means agent-based or synthetic transaction monitoring. The source of key metrics and data collection intervals were important considerations. Jim already had some network infrastructure monitoring tools, so he needed to consider how to integrate that information into the APM tool.

Most important to the CIO, the data presentation must include a way to quickly see the performance of the app and include out-of-the-box reports around the performance from the user perspective. While many APM tools have good dashboards, we shared with Jim that he might need another tool specifically for a dashboard, depending on the complexity of the visualization that links business data. The impact of a performance issue with a specific customer may require a dashboard with some intelligence.

Beyond technical requirements, we had a fourth section in the requirements document that dealt with training, online documentation, and support levels that will be required from the vendor Jim selects. We also took some time here to speak with Jim about the process needed to ensure the performance monitoring adapted to the changes that will occur with the application. Planned downtime was an example of this, as the tool needed to adapt to the company’s planned weekly maintenance window, and exclude performance data from that timeframe.

With the requirements complete, Jim now had to justify how much he could spend on the APM tool. His budget was tight and while this was a mandate from the CIO, he still needed to present a business case to the CFO. Jim was really concerned that his budget would not match his requirements. Building the business case for APM around revenue generating tools like an online ordering application is a bit easier than a billing application, because with online ordering it may be a bit easier to calculate revenue impact.

This is why Jim never had an APM tool in the past.  He knew there was value in APM, but it was hard to quantify. Jim did his best creating the business case and with the CIO, went to present to the CFO. If approved, Jim can then move to the next step of evaluating vendors. We’ll tell you how it goes next week.

In October of 2010, when we launched TRAC Research, we based our approach for covering IT performance management technologies on two advises that we were given by end-users:

• Don’t evaluate products by throwing them into technology buckets, but talk about what these products can do in specific usage scenarios

• Distinguish impactful from “cool” technologies, meaning discover what are the measurable business benefits from deploying a technology solution, not how “hot” the technology is

We thought that the best approach for doing this would be to launch an end-user survey and ask folks that are using this technology what their experiences are. This is when things started to get really messy. Before we even formulated the questions, we conducted close to 150 interviews with end-users, executives of technology vendors, prominent writers and some true thought leaders in this space to make sure that the questions are spot on to what they care about. Just to clarify, none of us are rookies in this space and for me, this is the 18^th survey of this type that I’ve created. Although, this time, launching the survey was more “interesting” than usual.

Creating the questions was easy, but defining the terms used in the questions was not easy at all. For example, one of the questions in the survey asked about capabilities that organizations could use when moving their services to the cloud. For that question, we had a set of prepared answer choices covering things like “automated process for adding or subtracting cloud resources” or “load testing of services that are being moved to the cloud”. When we delved more into it, we realized that whatever data that we would get from a question like that would be completely flawed and unusable for feature reports. That came as a result of the fact that there is no common understanding of what cloud computing really means to begin with. How can you normalize any data collected on trends in cloud computing if for some organizations cloud computing services means “any IT services that can be accessed through public Internet”, while for others it means “using a single physical server to host multiple applications and services”?

Also, we have been hearing that making IT performance data more actionable is one of the key goals for end-users. Well, we have also learned that for some folks that means “reduce the number of false alerts (or false positives)” while for others this means “difficulty to correlate performance data collected by different tools”. The same goes for concepts such as BSM, APM or even WAN optimization. Is BSM a management concept or a class of technology? What does end-to-end management of application performance really mean? Does WAN optimization mean accelerating and reducing network traffic or fully managing applications that are transferred over the WAN? Based on hundreds of conversations that we have had, the answer to all of these questions is the same, and it depends on whom you ask.

The key takeaway from these conversations with end-user organizations is that the lines between different classes of IT performance technologies are becoming very blurry and the business value of these solutions cannot be evaluated seperately from technologies that compliment them.

The mission of TRAC Research is not to redefine any of these technology groups or to create a new set of acronyms and/or buzzwords. What we can do is use this survey as a platform for end-users to report their experiences from using different technology capabilities and, in turn, report that back to the market. From there, it should be easier to understand the true value of IT performance management products, regardless of any technology bucket that they might be thrown into.

The survey is still open and you can access it here: 2010 IT Performance Management Survey

The survey is consisted of several sections, with each of them being specific to different flavors of IT performance management, such as business service management (BSM), network and application performance monitoring, virtualization and cloud management, Web performance, business transaction management (BTM), WAN optimization and performance testing. Respondants will only be asked questions from no more than two of these sections that are the most relevant to them.

All respondents will get more than 20 reports that we’ll publish from this survey for free.

Keep users happy with service-level agreements and application performance management.

Written By: Michael Biddick, InformationWeek
(Original Article)

Lots of IT organizations use an informal metric for measuring how satisfied employees are with application performance: If they aren’t calling the help desk, things must be OK. But this is less a strategy than an avoidance tactic. Proactive organizations will work with business units to create internal service-level agreements that define acceptable performance metrics. An SLA provides a formal mechanism for CIOs to report on progress in meeting performance goals–and to demonstrate to business leaders the very valuable services that IT provides.

Of course, if we’re going to offer SLAs, we must measure service levels with some kind of quantifiable metric. This is where application performance management comes in. APM tools are a vital source of information about the components that support an application, including software, server hardware, and network systems.

There are three basic steps to formalizing an internal SLA.

First, you need to define the metrics that will be used to calculate meaningful performance to the business; a promise of less than 10% packet loss is meaningless to the national sales manager. Focus on simple, yet telling, metrics such as user response time, the average time for a business unit to order IT services, or other customer-centric measures. Behind the scenes, these metrics will comprise a mix of transaction response times, CPU utilization, I/O responses, and so on.

Second, determine how you’ll enable business units to review SLA-related metrics. Will IT create regular reports to send to business leaders, or will information be provided via other means, such as an intranet? If you already have a corporate portal with scorecard metrics, you should be able to disseminate APM data there. If not, you may need to use a third-party management portal or an external reporting tool. In addition, many enterprise APM systems, including those from vendors such as BMC, CA, Compuware, and IBM, can present application service data to management.

Third, the IT group and business units must discuss how violations or potential violations will be reported and agree on consequences if IT fails to meet its agreements. For example, if a user phones the service desk to complain that an application is slow, should this kick off a process to investigate whether an SLA has been violated? Or should IT and business units meet regularly to examine performance numbers?

Behind the scenes, IT organizations that implement internal SLAs have to grapple with new technologies, such as cloud computing and virtualization. When it comes to public cloud services, including applications delivered in a software-as-a-service model, IT can’t get complete visibility into the provider’s infrastructure, complicating SLA measurement. As for virtualization, you need to ensure that your team can properly monitor and measure virtual environments. If IT can’t see performance issues in a virtual machine, it risks violating SLAs.

Find out more information on how you can keep your users happy with application performance management from Absolute Performance, Inc.

In a recent interview, Jimmy Harris, managing director of cloud computing at Accenture, a global management consulting, technology services and outsourcing company, shared his thoughts on how the job of IT management will change over time as enterprises take a more services-centric view of their worlds. His advice in a nutshell:

If you’ve got a penchant to know everything about anything, then you best shake it. And, likewise, polish your IT Infrastructure Library (ITIL)-like service management knowledge and business acumen.

Here’s why, he explains. As more robust cloud services bubble up, abstraction of infrastructure will become the IT endgame. “Enterprises won’t be managing infrastructure from an operational and service delivery perspective but essentially acquiring it as services,” he says.

As that happens — and certainly this won’t be an overnight occurrence but an evolutionary process — fewer bodies are needed for maintenance, enhancements and operations, Harris says.

From engagements at forward-thinking clients, what Accenture sees is a maturing of roles and technologies focusing on ITIL-like processes such as service management, governance, service integration, master data management and, newly emerging, service aggregation, Harris says.

“They’ve got to be able to work with third-party service providers for end-to-end performance management across a suite of technologies or applications. We’ll see an emphasis on supplier management, being able to manage the multiple suppliers that provide these various services in a way that makes sense to the business. And increasingly, ‘IT’ folks will need even more skills than do today in business process management, understanding the business in which they operate in and how technology can drive innovation in that business,” he explains.

And this doesn’t mean bringing in a new set of monitoring tools to manage the services, Harris adds. That’s not future-think.

“IT’s job will be to define the level of performance expected from service providers and be able to demonstrate that they’re in fact meeting those performance levels — but not necessarily doing the management and monitoring itself,” he says. “IT has to get out of the mindset that it’s going to create some sort of overarching and duplicative layer of delivery management because that’s what is done today.

“We have to get people out of the mindset of doing day-to-day operational delivery tasks and instead have them look forward and be able to say, ‘OK, I’m going to deliver this sales force application as a service — what does that mean and what pieces have to fit together? Do I source from a third party or internally to be able to deliver the amount of functionality and performance needed to meet the business requirements and, in turn, what does that mean?’”

It comes down to this: “IT will be the aggregator, assembler, tester and, to a certain extent, deployer of technology on behalf of the business, but won’t necessarily build or operate those technologies and applications itself,” Harris says.

“In a nutshell,” he adds, “IT will be more akin to being an uber integrator than to being a builder of specific components. That’s going to be important going forward.”

See how Absolute Performance can help your End-User Performance.

SaaS is slowly gaining adoption not just in the small and mid market range but also in big enterprises. According to a new Gartner report “Software as a Service 2009-2014″, the SaaS revenues within the enterprise software market will grow in 2010 by 14.1 percent from the 2009 revenues. In 2009, the revenues were $7.5 billion and it is growing to $8.5 billion this year.

A website or blog’s load time is pivotal to it’s success. Studies have shown that the highest cause of users abandoning a site is because it takes too long to load.

It’s a strange world we live in where patience is not the order of the day. We have faster computers, faster internet, better looking websites, interactive sites. Yet users still want that instant gratification.

It’s a never ending spiral. The more you put into your site to make it more attractive and interactive, the faster users expect it to load.

Google Incorporating page load into rankings

Google has incorporated site speed into their ranking algorithms. Although not a significant part, it is Google’s way of suggesting to us to get our sites to load faster.Google’s official blog post about site speed encourages users to test their website load times in order to evaluate and improve.

More than just page load times

Stress testing and load testing is more than just measuring the load times of a page. It’s about seeing how well a site and server operates under load.

When requesting a page speed test the results are normally shown on a 1:1 basis. That is how one user experiences your web page as far as speed is concerned.

But in the normal world we are not dealing with just one user. Stress and load testing sees how well you site does when multiple users browse multiple pages within a given period of time.

An average blogger might not be too concerned with stress testing at the moment. But if you ever desire to reach the kind of page views that a lot of the A-list blogger get, then you better be prepared.

With some reaching to about 100,000 page views a day. Which works out to just over 1 page every second.

The question is, can your current hosting, server, and application environment handle this kind of traffic?

If your website or blog is currently on a shared hosting package then this becomes even more important as your site or blog grows.

But what’s more important is that we do not know the load of the other 100’s of sites hosted on the same server are. They could well be pulling down your site speed.

What is website Stress Testing?

Most people think that throwing bandwidth at a site will sort out the speed problem. This is so untrue. Most site problems are as a result of servers, web servers, data servers, proxies, routers, etc, not being able to service the many requests per second.

You can have 100 gigs of bandwidth but a badly written application, an over loaded server will slow you down to less that a 1200 modem.

Look at what happened with the FIFA 2010 Soccer World Cup ticket sales. The system crashed time and time again because of being over stressed. Too many users buying World Cup tickets at the same time. I wonder how they stress tested their applications?

Web site stress testing tools simulate thousands of users accessing your website in a random fashion. It then determines how many concurrent users your site can handle, or how well your site performed under the given conditions.

Many website stress and load testing tools give an idea how your website will perform under given load parameters.

It is better for your website to fail under a simulated load test than when a user or potential customer is trying to access it.

Stress and load test tools generate simulated user traffic to your site. For example it might simulate that 50 users are trying to load your web pages at the same time. While simulating the traffic from these 50 users, it also records how fast pages are loaded from your server. This lets you know how fast your site is (as experienced by a user) when it is being accessed by 50 users at the same time.

For more information on Stress Testing and End-user Experience, check out Absolute Performance tools such as: StressWalk™ and WebWalk™.

​