Researchers
spend a great deal of time reading research papers. However, this skill is
rarely taught,
At
AMASA EDUCATIONAL SERVICES we always advice our students registered through us
for International Universities abroad, to learn to do so which will help you in
your University Education in future.
Follow
our blog regularly and we will publish a series of articles relating to your
educational preparations for future graduate studies. http://www.amasaeducationalservices.blogspot.com/
Later
in another article we will discuss about how to write a research paper. In this
article we consider how to read a research paper.
Researchers
must read papers for several reasons: to review them for a conference or a
class, to keep current in their field, or for a literature survey of a new
field. A typical researcher will likely spend hundreds of hours every year
reading papers. Learning to efficiently read a paper is a critical but rarely
taught skill. Beginning graduate students, therefore, must learn on their own
using trial and error. Students waste much effort in the process and are
frequently driven to frustration. Thus it is better to start reading now itself
so that you wont feel completely alien when you are asked to read one in you
lecture room when you start you Studies.
You
will need to carefully read a research paper if you are asked to review it, or
if it is relevant to your own research. We might also later discuss how to skim
a paper, so that you can decide whether a paper is worth a careful reading.
When
you read a research paper, your goal is to understand the scientific
contributions the authors are making. This is not an easy task.
It
may require going over the paper several times. Expect to spend several hours
to read a paper.
Here
are some initial guidelines for how to read a paper:
Read critically: Reading a research paper must be
a critical process. You should not assume that the
authors
are always correct. Instead, be suspicious.
Critical
reading involves asking appropriate questions. If the authors attempt to solve
a problem, are they solving the right problem? Are there simple solutions the
authors do not seem to have considered? What are the limitations of the
solution (including limitations the authors might not have noticed or clearly admitted)?
Are
the assumptions the authors make reasonable? Is the logic of the paper clear
and justifiable, given the assumptions, or is there a flaw in the reasoning?
If
the authors present data, did they gather the right data to substantiate their
argument, and did they appear to gather it in the correct manner? Did they
interpret the data in a reasonable manner? Would other data be more compelling?
Read creatively: Reading a paper critically is
easy, in that it is always easier to tear something down than to build it up.
Reading creatively involves harder, more positive thinking.
What
are the good ideas in this paper? Do these ideas have other applications or
extensions that the
authors
might not have thought of? Can they be generalized further? Are there possible improvements
that might make important practical differences? If you were going to start
doing research from this paper, what would be the next thing you would do?
Make notes as you
read the paper: Many people cover the margins of
their copies of papers with notes. Use whatever style you prefer. If you have
questions or criticisms, write them down so you do not forget them. Underline key
points the authors make. Mark the data that is most important or that appears
questionable. Such efforts help the first time you read a paper and pay big
dividends when you have to re-read a paper after several months.
It
would be easier if more research papers were well written... but again, we will
discuss writing later on.
After the first
read-through, try to summarize the paper in one or two sentences:
Almost
all good research papers try to provide an answer to a specific question.
(Sometimes the question is a natural one that people specifically set out to
answer; sometimes a good idea just ends up answering a worthwhile question.) If
you can succinctly describe a paper, you have probably recognized the question the
authors started with and the answer they provide. Once you have focused on the
main idea, you can go back and try to outline the paper to gain insight into
more specific details. Indeed, if summarizing the paper in one or two sentences
is easy, go back and try to deepen your outline by summarizing the three or
four most important subpoints of the main idea.
If possible, compare
the paper to other works:
Summarizing
the paper is one way to try to determine the scientific contribution of a
paper. But to really gauge the scientific merit, you must compare the paper to
other works in the area. Are the ideas really novel, or have they appeared
before? (Of course nobody expects you to be experts and know the areas ahead of
time)
It
is worth mentioning that scientific contributions can take on many forms. Some
papers offer new ideas; others implement ideas, and show how they work; others
bring previous ideas together and unite them under a novel framework.
Knowing
other work in the area can help you to determine which sort of contribution a paper
is actually making. For this you should keep on reading more & more
research papers which will improve your knowledge in the subject.
For many
years I have used a simple approach to efficiently read papers. This paper
describes the ‘three-pass’ approach and its use in doing a literature survey.
THE THREE-PASS APPROACH
The key idea
is that you should read the paper in up to three passes, instead of starting at
the beginning and ploughing your way to the end. Each pass accomplishes
specific goals and builds upon the previous pass: The first pass gives you a
general idea about the paper. The second pass lets you grasp the paper’s
content, but not its details. The third pass helps you understand the paper in
depth.
The first
pass :
The first pass is a quick scan to get a bird’s-eye view of the paper. You can also decide whether you need to do any more passes. This pass should take about five to ten minutes and consists of the following steps:
The first pass is a quick scan to get a bird’s-eye view of the paper. You can also decide whether you need to do any more passes. This pass should take about five to ten minutes and consists of the following steps:
1. Carefully read the title, abstract, and
introduction
2. Read the section and sub-section headings,
but ignore everything else
3. Read the conclusions
4. Glance over the references, mentally ticking
off the ones you've already read
At the end
of the first pass, you should be able to answer the five Cs:
1. Category: What type of paper is this? A
measurement paper? An analysis of an existing system? A description of a
research prototype?
2. Context: Which other papers is it related
to? Which theoretical bases were used to analyse the problem?
3. Correctness: Do the assumptions appear to be
valid?
4. Contributions: What are the paper’s main
contributions?
5. Clarity: Is the paper well written?
Using this
information, you may choose not to read further. This could be because the
paper doesn’t interest you, or you don’t know enough about the area to
understand the paper, or that the authors make invalid assumptions. The first
pass is adequate for papers that aren’t in your research area, but may someday
prove relevant. Incidentally, when you write a paper, you can expect most
reviewers (and readers) to make only one pass over it. Take care to choose
coherent section and sub-section titles and to write concise and comprehensive
abstracts. If a reviewer cannot understand the gist after one pass, the paper will
likely be rejected; if a reader cannot understand the highlights of the paper
after five minutes, the paper will likely never be read.
The second
pass :
In the second pass, read the paper with greater care, but ignore details such as proofs. It helps to jot down the key points, or to make comments in the margins, as you read.
In the second pass, read the paper with greater care, but ignore details such as proofs. It helps to jot down the key points, or to make comments in the margins, as you read.
1. Look carefully at the figures, diagrams and
other illustrations in the paper. Pay special attention to graphs. Are the axes
properly labeled? Are results shown with error bars, so that conclusions are
statistically significant? Common mistakes like these will separate rushed,
shoddy work from the truly excellent.
2. Remember to mark relevant unread references
for further reading (this is a good way to learn more about the background of
the paper).
The second pass should take up to an hour.
After this pass, you should be able to grasp the content of the paper. You
should be able to summarize the main thrust of the paper, with supporting
evidence, to someone else. This level of detail is appropriate for a paper in
which you are interested, but does not lie in your research speciality. Sometimes
you won’t understand a paper even at the end of the second pass. This may be
because the subject matter is new to you, with unfamiliar terminology and
acronyms.
Or the authors may use a proof or
experimental technique that you don’t understand, so that the bulk of the paper
is incomprehensible. The paper may be poorly written with unsubstantiated
assertions and numerous forward references. Or it could just be that it’s late
at night and you’re tired. You can now choose to: (a) set the paper aside,
hoping you don’t need to understand the material to be successful in your
career, (b) return to the paper later, perhaps after reading background
material or (c) persevere and go on to the third pass.
The third pass :
To fully understand a paper, particularly if
you are reviewer, requires a third pass. The key to the third pass is to
attempt to virtually re-implement the paper: that is, making the same
assumptions as the authors, re-create the work. By comparing this re-creation
with the actual paper, you can easily identify not only a paper’s innovations,
but also its hidden failings and assumptions. This pass requires great
attention to detail. You should identify and challenge every assumption in
every statement.
Moreover, you should think about how you
yourself would present a particular idea. This comparison of the actual with
the virtual lends a sharp insight into the proof and presentation techniques in
the paper and you can very likely add this to your repertoire of tools. During
this pass, you should also jot down ideas for future work. This pass can take
about four or five hours for beginners, and about an hour for an experienced
reader. At the end of this pass, you should be able to reconstruct the entire structure
of the paper from memory, as well as be able to identify its strong and weak
points. In particular, you should be able to pinpoint implicit assumptions,
missing citations to relevant work, and potential issues with experimental or analytical
techniques.
DOING A LITERATURE SURVEY
Paper reading skills are put to the test in
doing a literature survey. This will require you to read tens of papers,
perhaps in an unfamiliar field. What papers should you read? Here is how you
can use the three-pass approach to help. First, use an academic search engine
such as Google Scholar or CiteSeer and some well-chosen keywords to find three
to five recent papers in the area. Do one pass on each paper to get a sense of
the work, then read their related work sections. You will find a thumbnail
summary of the recent work, and perhaps, if you are lucky, a pointer to a
recent survey paper. If you can find such a survey, you are done.
Read the survey, congratulating yourself on
your good luck. Otherwise, in the second step, find shared citations and repeated
author names in the bibliography. These are the key papers and researchers in
that area. Download the key papers and set them aside. Then go to the websites
of the key researchers and see where they’ve published recently. That will help
you identify the top conferences in that field because the best researchers
usually publish in the top conferences.
The third step is to go to the website for
these top conferences and look through their recent proceedings. A quick scan will usually identify recent
high-quality related work.
These papers, along with the ones you set
aside earlier, constitute the first version of your survey. Make two passes through
these papers. If they all cite a key paper that you did not find earlier,
obtain and read it, iterating as necessary.
This disciplined approach prevents you from
drowning in the details before getting a bird’s-eye-view. It will allow you to
estimate the amount of time required to review a set of papers.
REFERENCES
[1] T. Roscoe, “Writing Reviews for Systems Conferences,”
http://people.inf.ethz.ch/troscoe/pubs/reviewwriting.pdf.
http://people.inf.ethz.ch/troscoe/pubs/reviewwriting.pdf.
[2] H. Schulzrinne, “Writing Technical
Articles,”
http://www.cs.columbia.edu/ hgs/etc/writingstyle.html.
http://www.cs.columbia.edu/ hgs/etc/writingstyle.html.
[3] G.M. Whitesides, “Whitesides’ Group:
Writing a Paper,”
http://www.che.iitm.ac.in/misc/dd/writepaper.pdf.
http://www.che.iitm.ac.in/misc/dd/writepaper.pdf.
[4] ACM SIGCOMM Computer Communication Review
Online,
http://www.sigcomm.org/ccr/drupal/.
http://www.sigcomm.org/ccr/drupal/.
By : G.A. Amodha Galgamuwa
