Doing a Quick Literature Review

Four ways to better exploit digital era capabilities

Image credit: CC0 Public Domain.

An elaborate literature review is an important stage in the development of almost all PhDs, and it is also a normal first step also in launching any new research project. There are two main versions.

Narrative reviews aim to give a ‘genetic’ account of the origins and development of understanding for a defined topic. They usually follow a basically chronological sequence — perhaps broken up into periods treated more as coherent wholes (‘periodization’); or perhaps analytically fragmented into component parts or sub-topics. The humanities and most of the social sciences are dominated by narrative reviews. Their proponents claim that they focus on ‘meanings’ and hence are especially appropriate for these human-focused and interpretative disciplines. Critics argue that narrative reviews are often partial, making no effort to be comprehensive. They are written up in ways that are qualitative or subjective; analysts rarely make explicit their criteria for assessment; and these evaluative criteria are not applied systematically. For some observers the limitations of narrative reviews are exposed by the wide gulf between the meager citations levels of the humanities and soft social sciences, compared with the far more extensive referencing included in STEM science papers.

Systematic reviews focus instead on results and attempt to find consensus (or at least an agreed picture) about effect sizes underlying apparently divergent or disparate findings. The analyst first explicitly defines a set of quality criteria to be used in comprehensively sifting through a large volume of literature. The criteria are used to progressively filter down the field of relevant work, so as to focus progressively on just the best-conducted studies. The analyst then seeks to condense out precise effect estimates of how a given cause or type of intervention A affects phenomenon X at the focus of analysis. Systematic reviews are highly developed in medicine, and they have spread into social sciences recently via the health sciences.

Critics argue that systematic reviews are most appropriate in fields where quantitative research predominates, where there is high consensus on problem definition, and where methods across studies are broadly comparable (rather than being contested). To be properly conducted systematic reviews also need to be comprehensive, which requires extensive searching in multiple databases. Systematic reviews are also tricky to do when a researcher’s understanding of issues and connections is not well developed. They need a thorough understanding of how problems fields relate to each other, which is inherently very difficult to acquire at the start of projects. Finally critics argue that a great many ‘dumb’ systematic reviews are now being churned out, using full text searches for precise word combinations in article or book titles. These may capture only a proportion of relevant literature, chiefly because academic researchers are endlessly adept at mis-describing their research in titles and in abstracts.

One of the greatest problems of large-scale and formalized literature reviews (both narrative and systematic) is that they take a long time to do. Hence researchers tend to be highly averse to repeating or renewing them — indeed with some systematic reviews the design may mean that ‘topping up’ at a later stage is not feasible. Yet over the course of a research project (and perhaps especially in a PhD project), the analyst’s understanding typically expands hugely. Anyone is far better informed on the realities of researching a given topic two or three years into it than one could hope to be at the outset.

Formal literature reviews may also get considerably over-extended by modern university practices. In the UK and Europe more ‘professionalized’ PhD training now often requires that doctoral students spend their first nine months (effectively a whole academic year) scoping their topic and conducting a literature review. There are precautionary institutional reasons why such a long period of navel-gazing and insubstantial, ‘throat-clearing’ or preliminary work are so commonly enforced. But this enforced childhood can have adverse effects on the developments of PhD students’ research. It tends to feed the illusion that problems’ solutions are to be found by an extended literature hunt, rather than by getting stuck into seriously trying to solve them for yourself. (As Schopenhauer famously said: ‘Do not read — think!’)

In research projects principal investigators are more experienced and tend to be quicker off the mark. But here too literature reviews often expand as a way of bringing new research staff up to speed. They also help construct an audit trail to convince grant-funders that no ‘duplicative’ work has been undertaken. Finally, of course, once the design of an experiment is fixed, and its equipment and protocols have been defined in a particular way, it is always tricky and may well then be impossible to adapt them or to do things differently. This strengthens the rationale for an exhaustive initial literature search to surface all options and help choose the best-adapted procedures. Yet in STEM sciences being the first to achieve and publish a given experimental result or breakthrough is of critical importance. So a huge ongoing amount of researchers’ time still subsequently has to be dedicated to monitoring and keeping up with current literature.

In the social sciences and humanities, by contrast, an initial literature review may well not be refreshed at later stages of long projects. It is quite common to see researchers looking surprised or even severely affronted when questioners at seminars or conferences, or even journal reviewers, ask that other literature or perspectives are taken into account. Such brush-off responses can suggest an entrenched unwillingness by investigators to consider literature not covered in their initial (often partial) review.

Digital literature reviews can be faster

We live now in a digital era, in which the idea of a giant initial literature review is of fading relevance, except for properly conducted systematic reviews. Instead of freezing our understanding of a field at one time, often indeed a time when we least understand the field, we should see the literature review as a repeated component of any ongoing research. We need more agile ways to surface other relevant research at every stage of our thinking and ‘writing up’, not just at the outset. We also need to consider how researchers actually work now, which is not very well presented by most institutional advice webpages or courses, generally produced by librarians I think, rather than by creative researchers themselves. So, in hopes that it will trigger some pushback comments and reactions, I set out here a first (deliberately controversial) attempt to outline strategies that contrast with the rather orthodox (and perfectionist) advice that seems to be out there at present.

1. Use Google search tools first and foremost. This may seem controversial to most librarians, who want researchers to use the proprietary bibliometic databases that they have expensively acquired, and sometimes researched about. But Google tools are clearly the best available in many dimensions and most disciplines, and they are easy to use in common ways, universally available on any internet PC or tablet, and free.

  • Google Scholar is by far and away the best (most inclusive) of the world’s bibliometric systems now, chiefly because it covers not just journal articles, but also citations of books and the many ‘grey literature’ reports originating from academic sources. You can search datewise in GS and it is usually fair enough to date restrict research searches to the last five or six years. GS also responds well to Boolean algebra search terms, putting linked terms in “double quotes”, and using operators such as AND, OR and NOT. You need to try a wide range of permutations of possible search terms, and to refine the combinations looked for in line with what the GS results are throwing up. Once you have well-defined search combinations, a realistic goal then is to make sure that you skim though the first 200 or 300 results. (If you set up your page to show 10 search results in snippet mode at a time then this is only 20 keystrokes). Typically there are likely to be three levels of results. (i) ‘Remote possible’ papers where you can just paste the GS ‘snippet view’ details into your ‘Materials’ file or archive, in case they might later prove interesting; (ii) Partly relevant papers, where you might copy across the article title and abstract only; and (iii) Clearly relevant papers, where you download the full text to your PDF library.
  • Google Books is an essential additional tool in the humanities and social science. Even in STEM disciplines it can be a useful add-on resource when seeking textbooks (best for explaining new materials), or the occasional ‘summation’, think piece or research commentary books from senior scientists. Essentially Google has now run around 10 million books through optical character readers so as to create online images of each page. For books that are out of copyright, Google makes available the full text for reading online, but the material cannot be downloaded in the free use version of the program. The text of most out of copyright books is also fully searchable, so you can easily find specific sentences, quotations, or words of interest anywhere in the book. For books still in copyright how much information is viewable on Google Books depends on what agreement the book’s publisher has reached with them. (i) The most restrictive ‘no preview’ option just replicates the publishers’ book blurb and perhaps gives the contents pages. (ii) The ‘snippet view’ offers only a few short glimpses of the book’s content. But this still allows readers to word search the full text for terms or phrases, and so assess how much coverage there is of relevant material. (iii) The most expansive Google Books preview shows many full pages of the text, but leaves out some key chapters or sections. However, you can still use the word search across these omitted sections, and get a snippet idea of what’s covered outside the full text pages. In either the snippet view or full text preview you can’t copy any text from Google Books. But in Windows, press Control+Print Screen to capture your screen view and then copy that image into a Word archive file. If a book covers what you are interested in only briefly, it’s easy to copy across these few pages of relevant text as a succession of images, obviating the need to consult the text itself. The text-finding software in Google Books is so powerful that many scholars now use it as an online index to find material within books already on their shelves, but which have either no index or the normally very inadequate academic book index system.
  • The main Google (Web) database has some specific search advantages compared to Scholar and Books, notably in being much more up to date, and in covering news media, blogposts, and the extensive ‘grey literature’ from corporate and professional bodies (as well as academic reports and working papers covered with a lag in GS). Because of its enormous size, however, main Google is best searched with relatively extensive and if possible distinctive phrases. For instance, searching for the author- and concept-distinctive jargon term ‘deliverology’ would be feasible, where the simpler and more ordinary language term ‘delivery’ would generate far too many entries to be useful. If your search initially throws up thousands of items, add additional content-distinctive words (again using Boolean operators) to get the numbers down to a feasible size. Similarly if you cannot find a quoted author or source, but you do have a quotation of at least five or six words, search for them inside “double quotes” and you’ll be surprised how quickly this thins down the lists of Google’s possibles.
  • Google Scholar Citations is a very helpful search extender. It’s a database of authors that most leading researchers now have an entry on, and which Google auto-updates so that it is always current. (If you’re a PhDer or researcher who’s not yet joined GSC, are you perhaps an academic hermit? Don’t be). Once you’ve got a list of ‘core’ articles or books directly relevant to your research from the sources discussed above, look up all the key authors on GS Citations to see if they have other publications on the same theme. GSC also shows you how many citations a given source has, so you can see roughly how important it may be. Sources that are heavily cited (given their age) generally deserve more care and attention. Often a major author may also have several versions of their argument, where generally the older one is more cited (because recent work takes quite a time to acquire citations). Outside STEM subjects key authors may well have both a book and article versions of the same work. Of course, different discipline groups also have varying citation rates, that you need to control for. Yet used intelligently citations levels help you cut through the problem of distinguishing remotely what is or is not likely to be important. GSC also lists all the other works in Google Scholar that have cited X’s key work (click on the cites number to see a list of these). For work directly related to your research, these listings have a high probability of including other relevant stuff, so it pays to search them fully and thoroughly. GSC also lists co-authors — so if you find that X has written well on your topic with Y, look up Y’s publications as well on GSC. For journal articles GSC has a powerful Google Scholar Metrics tool. Just enter the name of a journal you are unfamiliar with, and GSC will show you excellent indicators of its importance in its home field. Finally, the GS Alerts service provides excellent personalized updates to researchers in line with their publications, and with the kinds of authors they are following. Especially if you have published a lot (so that GS has a lot of information about your research interests to go on) these Alerts are phenomenally accurate and time-saving. They may not be so effective for new researchers though. Taken together, these tools help you do a digital equivalent of ‘searching along the shelf’ in the library, but in a much faster way and with far more useful contextual information.

2. Learning to use proprietary databases, such as the cross-disciplinary World of Science and Scopus, and more single-discipline or topic-focused resources , such as PubMed, can take time. These expensive and charged for databases are all human-compiled and so require that your university library has a subscription for you to gain access. They generally focus on articles in academic journals judged ‘reputable’ on conservative criteria. It can be time-saving and helpful to know that what elite Western academics might judge as ‘marginal’ work has mostly been excluded, but these databases still include a range of materials sufficiently wide to make it vital that you judge directly the quality of what you find. The biggest advantage of the proprietary databases is including new literature from core journals quickly, an especially important feature in STEM disciplines. Yet their ‘legacy’ designs often predated the modern digital era, making them ‘clunky’, quite difficult to use, and different from one to another. Because they are hard get familiar with initially, and re-familiarize with after a break, you normally have to be trained in the Library about how to use them creatively. So these systems tend to reinforce the idea of a literature review as a discrete phase of research.

Outside STEM sciences, the coverage of many edited databases like these is quite poor, with low ‘internal inclusion’ levels (an index showing how many references from articles included the database are also sources found within it). Not covering books and chapters in books is a big problem in most social sciences and all humanities disciplines. A literature review compiled on this basis might be worse than useless, because it is actively misleading. Scopus has included some books for some time, and Web of Science has been trying to reverse this blank spot recently. But neither comes close to the comprehensive coverage in Google products.

Similarly, if your research focuses on regions outside Europe, north America or Australasia, it is important to recognize that despite their size, the main proprietary databases cover only a small proportion of published work about these areas — perhaps as little as 3 per cent of all journal articles worldwide. They are also English-biased sources, which may matter a lot if a lot of research in your topic area is published in other languages.

3. Open access sources matter increasingly, chiefly because they give you quick access to full texts. They range now from ‘born open access’ journals (like plusone, now the biggest journal in the life sciences area); through web-based alerting and discussion paper series (like Arxiv in physics and neighbouring areas); over to leading blogs (such as EUROPP, a leading European blog curated by LSE). Archive sites that store full text versions of articles, chapters and papers are also very important, with ResearchGate running an especially good service that alerts you when another researcher you are following deposits any new materials. Twitter and Facebook streams linked to blogs, or based around other open access databases, also now serve as very important means of academic communication in the social sciences and humanities. Many older researchers have bewailed the increasing volume of academic work and their inability to keep pace with it. But the full range of search extension services should actually mean that you can keep up to date more easily across a far wider range of materials than in the past.

4. Store full texts digitally (and don’t make conventional notes for recall reasons). At research level, details and precision matter — for instance, how you design an experimental protocol; how strong different observed effects are; or how to detect and interpret complex multi-causation patterns. It’s no good making conventional notes about such things. If a source is directly relevant for your research, you need to store it securely on your own Cloud space or hard disc drive; you need to see exactly what the original author(s) said; you should comment directly on it so as not lose your reactions and questions; and you need to have both text and comments constantly available to you, to consult whenever you are citing or covering that source.

Modern bibliographic management tools also go far beyond the old confines of reference-only systems like EndNote and RefWorks. The package Zotero helps you clip, annotate and store all forms of material that you read digitally, annotate it and then manage your accumulating collection. Not everyone gets on with it though. Mendeley lets you store PDFs and other documents in along with details that are ready-formatted for re-expressing in bibliographies of different formats. You can also of course store all your own publications with them, and create groups that have collective resources. Each of these packages takes time and often training to understand, but for well organized people they should repay early learning and input costs with improved later access to sources.

For the social sciences and nearby humanities subjects, some related information for the social sciences can be found in : Simon Bastow, Patrick Dunleavy and Jane Tinkler, ‘The Impact of the Social Sciences ’ (Sage, 2014) or the Kindle edition. You can also read the first chapter for free and other free materials are here.

To follow up relevant new materials see also my stream on Twitter @Write4Research and the LSE Impact blog

find the cost of your paper

Sep 13, Grand Remembrances

Today is Grandparents Day in the United States. Being a Grand is a special honor. I feel very blessed that my wife and I have two grandchildren. We were able to visit them today. Yes, we are still being cautious with the coronavirus, but we also find it very difficult to not see them when they live so close. So today we did drop by to visit Jacob (age 10) and Sophia (age 7) along with their parents. We brought donuts and caught up with them. Our grandchildren are still pretty young and this is a precious time in their lives – and ours!

I wish I had known my grandparents better. We never lived in the same place. Dad was a career Air Force pilot, so we moved around a lot. But we did get to see them once in a while when they would visit us, or we them.

A Plague of Giants

There are five known magical ‘kennings’ or types: air, water, fire, earth, and plants. Each nation specializes in of these kennings, and the magic influences the society. There’s a big pitfall with this diversity of ability and locale–not everyone gets along.

Enter the Hathrim giants, or ‘lavaborn’ whose kenning is fire. Where they live the trees that fuel their fire are long gone, but the giants are definitely not welcome anywhere else. They’re big, they’re violent, and they’re ruthless. When a volcano erupts and they are forced to evacuate, they take the opportunity to relocate. They don’t care that it’s in a place where they aren’t wanted.

I first read Kevin Hearne’s Iron Druid books and loved them (also the quirky The Tales of Pell), so was curious about this new venture, starting with A PLAGUE OF GIANTS. Think Avatar: The Last Airbender meets Jim Butcher’s Codex Alera series. Elemental magic, a variety of races, different lands. And it’s all thrown at you from page one.

But this story is told a little differently. It starts at the end of the war, after a difficult victory, and a bard with earth kenning uses his magic to re-tell the story of the war to a city of refugees. And it’s this movement back and forth in time and between key players in this war that we get a singularly grand view of the war as a whole. Hearne uses this method to great effect.

There are so many interesting characters in this book that I can’t cover them all here. Often in books like this such a large cast of ‘main’ character can make the storytelling suffer, especially since they don’t have a lot of interaction with each other for the first 3/4 of the book–but it doesn’t suffer, thankfully. And the characterization is good enough, despite these short bursts, that by the end we understand these people and care about what happens to them.

If there were a main character it would be Dervan, a historian who is assigned to record (also spy on?) the bard’s stories. He finds himself caught up in machinations he feels unfit to survive. Fintan is the bard from another country, who at first is rather mysterious and his true personality is hidden by the stories he tells; it takes a while to understand him. Gorin Mogen is the leader of the Hathrim giants who decide to find a new land to settle. He’s hard to like, but as far as villains go, you understand his motivations and he can be even a little convincing. There’s Abhi, the son of hunters, who decides hunting isn’t the life for him–and unexpectedly finds himself on a quest for the sixth kenning. And Gondel Vedd, a scholar of linguistics who finds himself tasked with finding a way to communicate with a race of giants never seen before (definitely not Hathrim) and stumbles onto a mystery no one could have guessed: there may be a seventh kenning.

There are other characters, but what makes them all interesting is that they’re regular people (well, maybe not Gorin Mogen or the viceroy–he’s a piece of work) who become heroes in their own little ways, whether it’s the teenage girl who isn’t afraid to share vital information, to the scholars who suddenly find how crucial their minds are to the survival of a nation, to the humble public servants who find bravery when they need it most. This is a story of loss, love, redemption, courage, unity, and overcoming despair to not give up. All very human experiences by simple people who do extraordinary things.

Hearne’s worldbuilding is engaging. He doesn’t bottle feed you, at first it feels like drinking from a hydrant, but then you settle in and pick up things along the way. Then he shows you stuff with a punch to the gut. This is no fluffy world with simple magic without price. All the magic has a price, and more often than not it leads you straight to death’s door. For most people just the seeking of the magic will kill you. I particularly enjoyed the scenes with Ahbi and his discovery of the sixth kenning and everything associated with it. But giants? I mean, really? It isn’t bad enough fighting people who can control fire that you have to add that they’re twice the size of normal people? For Hearne if it’s war, the stakes are pretty high, and it gets ugly.

The benefit of the storytelling style is that the book, despite its length, moves along steadily (Hearne is no novice, here). The bits of story lead you along without annoying cliffhangers (mostly), and I never got bored with the switch between characters. It was easy to move between them, and they were recognizable enough that I got lost or confused. The end of the novel felt a little abrupt, but I guess that has more to do with I was ready for the story to continue, despite the exiting climax.

If you’re looking for epic fantasy with fun storytelling and clever worldbuilding, check out A PLAGUE OF GIANTS.

The post A Plague of Giants appeared first on Elitist Book Reviews.

The Artwork Of Gary Choo

Gary Choo is a concept artist/illustrator based in Singapore. I’ve know Gary for a good many years ( 17, actually ), working together in animation studios in Singapore like Silicon Illusions and Lucasfilm. Gary currently runs an art team at Mighty Bear Games, but when time allows he also draws covers for Marvel comics, and they’re amazing –

The Art Of Gary Choo
The Art Of Gary Choo
The Art Of Gary Choo
The Art Of Gary Choo
The Art Of Gary Choo

To see more of Gary’s work or to engage him for freelance work, head down to his ArtStation.

The post The Art Of Gary Choo appeared first on Halcyon Realms – Art Book Reviews – Anime, Manga, Film, Photography.