PR-Do you know what are they saying about you? (sic)

The following information is used for educational purposes only.

Do you know what are they saying about you?(sic)

The challenge of online monitoring


James L. Horton and Shade Vaughn

CEOs don’t like surprises. They don’t want to learn about
customer gripes after customers have departed. They don’t
want to find out that employees are bad-mouthing the
company after comments have affected the stock price. They
definitely do not want rumors about their companies, their
employees or themselves spread around the world and be the
last to know it is happening. They are irate when inside
information about competitors and the industry is bandied
about before they hear it. But this is what can happen on the
Internet and Web daily.
The Internet is an international, round-the-clock bull session
splintered into tens of thousands of newsgroups and Web site
bulletin boards. Some people actively participate in
discussions and some observe without posting comments but
are influenced just the same. Some discussions are influential.
Some are gripe fests. Some wither while others emerge with
new topics and participants. Company executives who think
they know what is being said online because they have
customer service systems, such as an 800- dial-in number,
mislead themselves. Service systems can be the problem. We
are aware of one company that learned from online monitoring
that its 800- dial-in service system was the largest source of
complaints. Customers were often enraged by what happened
to them when they called the company. They vented their
rage on bulletin boards. CEOs concerned about what
customers, competitors and others say need to know if their
companies, products and services are discussed online and
what is said.
Unfortunately, this is difficult to do. It is hard to monitor
online discussions because bulletin boards are moving from
well-known and well-documented Internet Usenet newsgroups
to tens of thousands of undocumented Web-based bulletin
boards. Secondly, Web-based bulletin boards use a variety of
software systems. The different systems confuse software
filters and spiders and, as a result, they fail.
This paper discusses what our agency has learned about
Internet and Web bulletin board monitoring from many
months of doing it. Our conclusion is that monitoring requires
good judgment and strong control to get results that CEOs and
other executives can use.

To monitor or not?
Some companies are talked about constantly on the Internet
and Web. Some are not. One cannot tell whether an industry
and/or company is a topic of conversation without going
online and looking. We know from experience that certain
industries are more likely to be discussed than others –e.g.,
consumer electronics companies, healthcare providers/issues
(diabetes, cancer, etc.), public companies on investor bulletin
boards and entertainment companies. But, this does not mean
a company in a different industry is unlikely to be talked
about. There is an equal chance that it might be because the
Internet and the Web gather interests and enthusiasms from
around the earth and from different socioeconomic levels and
cultures.
Even if a company is not talked about today, there is no
guarantee it will not be talked about tomorrow. Bulletin board
topics change as quickly as human attention. We recommend
even to companies that do not need to monitor online bulletin
boards today to do an annual checkup to see what is being said
about their industries, competitors and themselves. Public
companies, especially, need to know what has been said about
them on investor bulletin boards.

Leading or lagging?
Some assume that bulletin board discussions lead opinion and
perception, but this is not true. Bulletin board discussions are
anachronistic. They exist outside of or alongside of time and
events in their own peculiar way. Urgent topics that show in a
company’s service system may not find their way to a bulletin
board. Other topics that never show in the company’s service
system may be the subject of raging arguments on bulletin
boards. Some topics lead a company’s service system and
some lag. For example, a company apprised us of a product
defect it was handling through its service system well before
discussion of the defect showed in bulletin boards. When the
topic did show, the writers seemed to accept that everyone on
the board already knew about the defect.
We have examined this baffling lack of chronology, but we
have no definitive answer for why it occurs. We have
hunches:
Online bulletin boards do not exist apart from other
media. Reports elsewhere may or may not spill into a
bulletin board depending on a writer’s interest or need.
For example, a product recall might be sufficiently
discussed in general news media that no one discusses
it on a bulletin board.
Bulletin board participants do not reflect a statistical
cross-section of a company’s key audiences. They are
a subset of company customers, investors, employees,
competitors and others who are comfortable engaging
in discussions on bulletin boards. Sometimes this
subset is large and sometimes not.
Bulletin board writers discuss what is important to
them and that may not be what is important to a
company. It is not uncommon for a company and its
audiences to have divergent interests. What a CEO
needs to know is whether the interest gap is harming
the company in some way or an opportunity the
company should seize.
Bulletin boards are to some degree a parallel universe
in which a company can find unusual views about its
industry and itself, some of which are enlightening and
some of which are crackpot.

Where Do I Find It?
The question of where to look on the Web for what might be
discussed about an industry, company or company executives
is huge. Finding discussion groups can be as difficult as
finding one title in a library without an index. Imagine
entering the Library of Congress with its millions of books
thrown into a vast heap on the main floor. The one book you
need is in the heap somewhere. Online, there is no compelling
reason, structure or trend for why a topic is discussed at this
site rather than that one or why it is discussed at all.
However, we recommend a disciplined approach to find out
what might be on the Internet or Web because without it one
can waste time with no result. We have learned to always
work from the easiest to hardest search method. Our first step
is to look at Usenet newsgroups. These bulletin boards exist
on the Internet and not the Web. They pre-date the Web, and
they all use the same methodology. That means they can be
tracked in databases easily – and fortunately, they are. The
best-known database of Usenet newsgroup bulletin boards is
Google Groups (formerly, deja.com) (www.google.com).
Using Google Groups, one can enter company or product
names and see what, if anything, comes out of its Usenet
search engine. Google Groups, however, is insufficient to
determine whether one should monitor or not because of the
large number of Web-based bulletin boards today.
One can “google” a company’s name or products in the Web
search section of Google’s search engine, but for many
reasons this works poorly. Much of what surfaces will have
no relationship to bulletin board conversations. In the second
search step, we favor searching Web sites in the industry, such
as sites operated by industry publications. Often, publications
will have bulletin boards, but just as often, the bulletin boards
are fallow. If one is lucky, an industry trade publication will
have an active board that can lead to other findings. For
example, it is not uncommon that a bulletin board member
will cite a comment on another bulletin board. Following the
comment can lead one to boards that are otherwise invisible.
During the second search phase, we also recommend
surveying company employees to see if they are aware of
bulletin boards that apply to the industry or company. This is
usually productive. Finally, for investor comments, we look at
well-known boards run by Yahoo!, for example.
If all this fails, brute force searching using engines like
Google or FAST may be the only avenue. Such searches are
tedious and often fruitless, but sometimes, they can crack the
riddle of whether and where people are discussing an industry
and companies in it. We recommend phrase searching.
Rather than having a search engine look on one word, we have
search engine look for a two- or three-word phrase. Phrases
cut down on the number of returns and increase the likelihood
of finding useful information. The trick is to know the phrase
to use. One needs sufficient knowledge of an industry and
company to choose phrases with the highest probability of
return. We can attest, however, that we have wasted hours on
searching phrases and variations on phrases without success.
However, here too, we have been lucky and found new
approaches. For example, an unproductive phrase sometimes
surfaces on a Web page that contains a new phrase, which
scores dozens of hits.
At some point in the search for boards, it is no longer
economic or efficient to continue. One should stop and
evaluate what has been found and decide whether it is worth
monitoring or not. We recommend setting a time and budget
limit in advance to prevent uncontrolled surfing. There is
something about the Internet and the Web that leads one on
and on.

How often should one monitor?
There is no answer to this question. Monitoring frequency
varies by the importance of the issues discussed. Some
companies monitor daily, some weekly, some bi-weekly, some
monthly and some at longer intervals. Obviously, in a crisis, a
company should track discussions in real-time, if bulletin
board discussions relate to the crisis. Sometimes, frequent
monitoring is called for during new product introductions to
see how the product/service is being treated. On the other had,
our experience has been that bulletin boards have already
surmised and discussed many features, functions and benefits
of pending new products before they are introduced. Bulletin
boards are notorious for scooping company marketing plans,
particularly if a company stages product introductions. For
example, a company might introduce a product in one country
then plan to roll it out later to another country. In the
consumer electronics industry, we have learned, this hardly
ever works. Bulletin board writers from the country where the
product was launched busily describe the product and its
features, functions and benefits to board participants
worldwide.
Two other times require frequent bulletin board monitoring –
when hot issues are discussed and when harmful rumors
plague a company. Hot issues can be as varied as discussion
of evolving industry standards to emerging competitive
threats. Some industries, such as networking and computing,
maintain constant bulletin board monitoring because there are
always hot issues. Other industries rarely face volatile
discussions. Rumors are dangerous, especially when they can
cause direct harm. This is a lesson that CEOs learned during
the dot-com era when stock hucksters and shorts would get on
a board and bash a company’s stock in order to watch it climb
or fall.

Who should monitor and be monitored?
These are critical questions for the success of an online
monitoring program. The person(s) who perform online
monitoring need an ability to understand what is being
discussed, to judge its usefulness and to collect data. These
are different skills. Understanding and judgment require an
individual who grasps enough of a business to know what
executives need to know. Data collection requires an
individual with the physical dexterity to do the boring task of
data collection and entry.
The reason that intelligent data collection is necessary is to
avoid the masses of junk that online discussion boards carry.
No monitoring service can afford to collect everything and if it
did, raw data is not useful to executives. Monitoring must
present the right data in the right way. Even though one can
train an individual to recognize and collect categories of
information, there is still need for editing to pare data to
insights that executives need. Monitoring can be split between
the collection and editing function, but not completely.
Finding individuals with the right combination of intellect and
physical ability often requires modification of the monitoring
process to keep a system effective.
Monitoring may not focus on any one individual, but there
are times when it should. For example, well-known and
influential critics of a company, its products and services
should be watched closely. These are individuals who may be
named or who write under a “screen handle” and have a
significant audience that follows their lead. These individuals
usually dominate a particular discussion board, and they may
even run it. Particularly if the media pay attention to what the
individuals write, one should monitor their opinions regularly.
A second group that should be watched are “High-frequency
responders.” These are individuals who do not dominate a
bulletin board but respond often enough that they have a
disproportionate share of opinions. Sometimes these opinions
can bias monitoring. It looks like more people are angry than
are, for example. The way to understand whether one is
dealing with a high-frequency responder is to perform a
statistical analysis of responses. This was the approach we
used, for example, to determine who was behind a customer
revolt fomented on bulletin board. A statistical sample of the
fulminations revealed four individuals behind most of the
comments. These individuals also were actively soliciting the
media to come to their aid. We were able to recommend
specific steps to the client when we discovered this, even
though these individuals were hiding behind “screen handles.”

How do I monitor?
Monitoring is custom-tailored to need. There is not one way
to do it. In situations where there are only a few dozen
comments a month, one can sample opinion and write a
summary report. In cases where there are hundreds or
thousands of comments monthly, a database is essential.
Where issues diverge from one division to the next in a
company, decentralized monitoring might be best. In other
cases, centralized monitoring is more efficient. Where the
department responsible for monitoring does not have or cannot
easily organize the resources, outsourcing is more efficient.
Otherwise, an internal monitoring department is highly
effective.
In most cases, monitoring is “lurking” – that is, the
monitoring department records and reports comments.
Sometimes, monitoring is interactive: Monitors respond to
comments with pre-approved comments. In a few cases,
monitors go undercover and do not reveal their identity while
participating on bulletin boards. This is a risky approach,
however, that can be unethical, and it requires great care. We
could not recommend it to any but the most skilled
practitioners. If bulletin board participants discover a
company representative undercover, it can touch off flame
wars, bad feelings and a loss of company reputation.
Monitoring is about dollars and cents. It should not cost
more to monitor than the value received from doing it. This is
not an easy calculation for perception is a large part of
valuation. If executives ignore data retrieved, monitoring has
little value, even if the information provides critical clues to
what the company should do. If executives consider
monitoring critical, they might well spend more than it is
worth to conduct it. Manager(s) in charge of monitoring bear
the responsibility for ensuring that the right data is presented
in the right way to the right persons and that these persons
understand its implications. When this is done effectively, the
value of monitoring (or lack of it) surfaces quickly.

Databases
Because we handle large volumes in monitoring, we use a
custom-configured relational database as our core tool. We
have learned through using it that it is not always clear how
one should build and operate a database, and there are many
considerations that enter into a final configuration. Here are
just a few:
What to database and what not. There is a lot of
information in any bulletin board message. There is
the name of the newsgroup or Web bulletin board; the
header that reveals where the message came from, its
date and who sent it and the subject line that indicates
the topic of the message. There is the message itself,
which may discus more than one topic. There are
categories of products and services that might be
discussed. There are opinions that might need rating.
Finally, there is the verbatim of the message that one
may or may not collect. One may need to database all
of these things or none. The tradeoff is in time and
data entry cost. The more one collects, the greater time
and cost it takes to monitor. The less one collects, the
less time and money it takes, but the chances of
missing critical information rise. Learning what to
collect and how is a matter of experience and close
attention to a client’s need. As a rule of thumb, it is
better to start by collecting too much than too little.
One can pare data later on when it is clear that it is no
longer relevant. On the other hand, it is hard to add
data that was left out because it is difficult to relocate
messages and database structures must be reconfigured.
Building a database. If one is not familiar with
building a relational database, the task can intimidate.
It is a job best left to professionals. On the other hand,
there are databases that make the job of building fields,
queries, data entry screens and reports easier than it
used to be. For small to medium-scale monitoring,
these might be sufficient for the job at hand. Whether
one builds a database personally or has it built, it is
essential that the person in charge of monitoring be
intimately involved with the development process.
Databases are only as good as the reports they
generate. A database that cannot produce the report
that users want is useless to them and ultimately, to the
monitoring process. The irony of database
development is that it takes familiarity with report
generation to understand what can be derived. This
only emphasizes the need to have professional help
when building one.
Usability. Usability is critical in database
development. Monitoring stays within budget only on
the basis of how one monitors. For every bulletin
board to scan, for every box to fill in, for every
keystroke the operator makes, there is a time cost that
translates to dollars. We recommend that monitoring
be reduced as far as possible to a point-and-click
exercise. Operators should be able to fill a screen in
about 20 seconds through clicking boxes and copying
and pasting appropriate items. This also reduces
operator burnout when dozens of comments are
collected daily. The greatest amount of time in
monitoring should be reserved for newsgroup and
bulletin board scanning. This is plowing through the
screens looking for what people are writing about.
Web-based bulletin boards resist efficiency in how they
are structured and operate even if they have good
search engines. Hence, operators must be given the
time to get through them without coming under
pressure from data entry.
Operator training. Operator training is critical to
effective monitoring. The operator must know what to
look for and how to enter it quickly. There is an
experience curve in monitoring. Operators start slowly
and accelerate over time as they learn the bulletin
boards they are monitoring, the topics likely to be
discussed and ways to enter data more quickly. Any
monitoring process needs to factor the experience
curve into budgeting, especially if the exercise is new.
For that reason, we recommend that monitoring
supervisors perform entry at least at the beginning of
the process to understand problems that operators are
likely to confront. This can quickly surface hidden
mistakes such as the sequence of data entry. Data
should be entered as the operator sees it on the bulletin
board, or entry dramatically slows and increases
burnout. We have found that operators will not
necessarily tell you that sequencing is out of order
because they assume the monitoring supervisor wants it
that way.
Statistics and metrics. Monitoring is about
measurement. Comments help little unless packaged in
ways that executives can quickly grasp if there is an
increase or decrease in concern about specific topics
and issues. Hence, monitoring reports should include
graphs of numbers of comments and scale ratings about
the comments themselves. Reports also should include
where the most comments are coming from and what
issues are most frequently brought up. The hidden
advantage of such metrics is they reveal to the
monitoring supervisor when monitoring might be going
off track. For example, 50 comments on a specific
topic in one period that suddenly drops to 20 the next
period is a red flag to the supervisor that the operator
has missed comments or something has happened
within the bulletin boards. The supervisor can then
check quickly for the source of the drop before it is
questioned elsewhere. In fact, metrics have proven
more useful to us in conducting monitoring than even
to clients who use them.

Summary
Many CEOs continue to fly blind when it comes to
monitoring online newsgroups and Web-based bulletin boards.
They seem to be of the opinion that what they don’t know
won’t hurt them and that whatever is said on a bulletin board
has little relevance unless it affects their stock price. This is a
dangerous dismissal of opinion that can undermine a
company’s reputation and ability to operate. Certainly, some
bulletin board writers are crackpots, but some are perceptive.
They are concerned about industries, companies, products,
services, pricing and a host of other issues that determine a
company’s competitive stance in the marketplace. It is
foolhardy to ignore them. The challenge to monitoring,
however, is to exercise judgment and control to get useful
results efficiently, quickly and within budget.

Jim Horton developed the online monitoring business and
technologies for Robert Marston And Associates, Inc. Horton
is the founder, developer and owner of www.online-pr.com, a
resource site for PR practitioners. Shade Vaughn manages
the online monitoring business for Robert Marston And
Associates.


Robert Marston And Associates, Inc.
Copyright, 2002. James L. Horton


Source: www.online-pr.com