002-scholarref.txt (8684B)
1 2 ## Rationale 3 During the writing phase of an academic paper, common tasks include 4 downloading PDFs of publications and getting their references into 5 your bibliography. However, I am not a fan of navigating the slow, 6 bloated, tracker-filled, and distracting webpages of academic 7 journals and publication aggregators. For some reason, many 8 publishers decided that clicking the "Download PDF" link should 9 redirect the user to an unusable in-browser PDF viewer instead of 10 providing the PDF file directly. While the majority of journal 11 webpages provide formatted citations for their publications, these 12 are inconsistent in style and content. 13 14 For these reasons, I constructed a set of shell tools called 15 "scholarref" that allow me to perform most of the tasks without 16 having to open a web browser. As the title of this post indicates, 17 the goal of the toolset is to provide as much functionality a person 18 might need during scientific writing without leaving the command 19 line. The tools are under continuous development[0]. At present 20 I avoid roughly 90% of visits to journal webpages. I hope to get 21 to 100% someday. 22 23 The "scholarref" design goals are the following: 24 25 - Written as POSIX shell scripts with minimal external 26 dependencies: Ensures maximum flexibility and portability. 27 28 - Aim for simplicity: 29 Fewer lines of code make the programs easier to understand, 30 maintain, and debug. 31 32 - Each tool should do one thing, and do it well: 33 Let the users piece the components together to fit their 34 workflow. 35 36 - Return references in BibTeX format. 37 38 DISCLAIMER: The functionality provided by these programs depends 39 on communication with third party webpages, which may or may not 40 be permitted by law and the terms of service upheld by the third 41 parties. What is demonstrated here are examples only. Use of the 42 tools is entirely your own responsibility. 43 44 45 ## Installation 46 47 $ git clone git://src.adamsgaard.dk/scholarref 48 $ cd scholarref 49 # make install 50 51 The `make install` command may require superuser priviledges to 52 install the tools to /usr/local. Prefix with doas(1) or sudo(1), 53 whatever is appropriate for the target system. 54 55 56 ## The scholarref toolset 57 58 The core functionality is provided by the scripts getdoi, getref, 59 and shdl. All programs accept input as command-line arguments or 60 from standard input (stdin). The programs come with several OPTIONS, 61 and it is encouraged to explore the help text (invoke with option 62 -h). The -t option may be of particular interest, since it tunnels 63 all communication through Tor via torsocks(1) (if available on the 64 system). 65 66 ### getdoi 67 This tool accepts either names of PDF files or arbitrary search 68 queries. If a PDF file name is supplied, getdoi scans the PDF text 69 in order to find the first occurring DOI entry, which typically is 70 the DOI of the publication itself. If an arbitrary query is supplied, 71 the CrossRef API[1] is used to find the DOI of the closest publication 72 match. You can supply author names, parts of the title, ORCID, 73 journal name, etc. Examples: 74 75 $ getdoi damsgaard2018.pdf 76 10.1029/2018ms001299 77 $ getdoi 'damsgaard sergienko adcroft advances modeling earth systems' 78 10.1029/2018ms001299 79 80 The -o option will open the resultant DOI in the system web browser. 81 82 ### getref 83 The getref tool fetches the BibTeX citation for a given DOI from 84 doi.org. By default, the journal names and author first names are 85 abbreviated, which is what most journals want. I have taken 86 abbreviations from the Caltech Library list of Journal Title 87 Abbreviations[2]. The getref ruleset of journal-title abbreviations 88 is incomplete, and is expanded on a per-need basis. If desired, 89 the abbreviation functionality can be disabled. See `getref -h` 90 for details. 91 92 $ getref 10.1029/2018ms001299 93 @article{Damsgaard2018, 94 doi = {10.1029/2018ms001299}, 95 year = 2018, 96 publisher = {American Geophysical Union ({AGU})}, 97 volume = {10}, 98 number = {9}, 99 pages = {2228--2244}, 100 author = {A. Damsgaard and A. Adcroft and O. Sergienko}, 101 title = {Application of Discrete Element Methods to Approximate Sea Ice Dynamics}, 102 journal = {J. Adv. Mod. Earth Sys.} 103 } 104 105 Do not abbreviate journal title with -j option: 106 107 $ getref -j 10.1029/2018ms001299 108 @article{Damsgaard2018, 109 doi = {10.1029/2018ms001299}, 110 year = 2018, 111 publisher = {American Geophysical Union ({AGU})}, 112 volume = {10}, 113 number = {9}, 114 pages = {2228--2244}, 115 author = {A. Damsgaard and A. Adcroft and O. Sergienko}, 116 title = {Application of Discrete Element Methods to Approximate Sea Ice Dynamics}, 117 journal = {Journal of Advances in Modeling Earth Systems} 118 } 119 120 121 ### shdl 122 This tool takes a DOI as input and attempts to download the 123 corresponding publication as a PDF through sci-hub[3]. Unfortunately, 124 the sci-hub web interface often puts up captias to restrict automated 125 downloads. If that's the case, shdl opens the tor browser (if 126 installed) or the system web browser in order to manually complete 127 the download. Output PDF files are saved in the present working 128 directory. 129 130 131 ## Usage examples 132 133 The scholarref tools are meant to be chained together. For example, 134 if you want a BibTeX reference a search query, simply use UNIX pipes 135 to send the getdoi output as input to getref: 136 137 $ getdoi 'damsgaard egholm ice flow dynamics' | getref 138 @article{Damsgaard2016, 139 doi = {10.1002/2016gl071579}, 140 year = 2016, 141 publisher = {American Geophysical Union ({AGU})}, 142 volume = {43}, 143 number = {23}, 144 pages = {12,165--12,173}, 145 author = {A. Damsgaard and D. L. Egholm and L. H. Beem and S. Tulaczyk and N. K. Larsen and J. A. Piotrowski and M. R. Siegfried}, 146 title = {Ice flow dynamics forced by water pressure variations in subglacial granular beds}, 147 journal = {Geophys. Res. Lett.} 148 } 149 150 The scholarref program itself is an aggregation of the getdoi and 151 getref commands. If called with the -a option, the reference is 152 directly inserted into the system bibliography. The full path to 153 the bibliography file (.bib) is assumed to be set in the $BIB 154 environment variable, for instance defined in the user ~/.profile. 155 156 $ echo $BIB 157 /home/ad/articles/own/BIBnew.bib 158 $ scholarref -a 'damsgaard egholm ice flow dynamics' 159 Citation Damsgaard2016 added to /home/ad/articles/own/BIBnew.bib 160 161 162 ## Integrating into your favorite $EDITOR 163 164 The scholarref tool is particularly useful if called from within a 165 text editor. Below I demonstrate how keyboard bindings can be bound 166 in various editors to provide scholarref functionality. 167 168 ### vi 169 My editor of choice is the plain, old, and simple (n)vi(1). I have 170 the following binding in my ~/.exrc, including a trailing space: 171 172 map qr :r !scholarref 173 174 The rest of my editor configuration can be found under my dotfiles 175 source code repository[4]. 176 177 ### vim 178 You can add the following bindings to ~/.vimrc or ~/.vim/vimrc in 179 order to get scholarref functionality within vim(1): 180 181 " insert reference into current buffer 182 nnoremap <leader>r :r !scholarref<space> 183 " append reference into $BIB file 184 nnoremap <leader>R :r !scholarref --add<space> 185 186 ### vis - 187 The vis(1) editor[5] is an interesting combination of modal editing 188 and structural regular expressions from the plan9 editor sam(1)[6]. 189 Add the following binding to ~/.config/vis/visrc.lua: 190 191 vis:map(vis.modes.NORMAL, leader..'r', '< scholarref ') 192 193 ### emacs 194 Don't know, figure it out yourself. 195 196 197 ## Integrating into your pdf viewer 198 199 My PDF viewer of choice is zathura(1)[7], which has a minimal 200 graphical user interface and is keyboard-centric. The following 201 configuration calls getdoi on the currently open file if I press 202 Ctrl-i. The resultant DOI is copied to the clipboard. Similarly, 203 Ctrl-s tries to extract the DOI in the same manner, but fetches the 204 accompanying reference and adds it directly to the bibliography. 205 206 map <C-i> feedkeys ":exec getdoi --notify --clip '$FILE'<Return>" 207 map <C-s> feedkeys ":exec scholarref --add '$FILE'<Return>" 208 209 My full zathura configuration is available here: [8] 210 211 212 ## Questions/bugs/feedback/improvements 213 214 Please get in touch if you encounter any. Improvement suggestions 215 are best sent as patches by e-mail. 216 217 218 References: 219 220 [0] https://src.adamsgaard.dk/scholarref/log.html 221 [1] http://api.crossref.org 222 [2] https://www.library.caltech.edu/journal-title-abbreviations 223 [3] https://sci-hub.tw 224 [4] https://src.adamsgaard.dk/dotfiles/file/.exrc.html 225 [5] https://github.com/martanne/vis 226 [6] https://sam.cat-v.org/ 227 [7] https://pwmt.org/projects/zathura 228 [8] https://src.adamsgaard.dk/dotfiles/file/.config/zathura/zathurarc.html