As the long-term readers among you might know, I am using EndNote for my citation and bibliography needs. Theoretically, I also have all the .pdf files that I have of articles in one folder on my system, and all the titles of these articles in the EndNote database.

Theoretically. Which, unfortunately, is not at all what I really have - over time, a large stack of .pdfs has accumulated that are not yet in the database, because it all meant opening the file, copying the relevant bibliographical data into the database, then saving the file under a new (and unique) name also written to the database file. Yes, the programme can theoretically attach files to database entries - but I have never gotten the hang of that, and I also prefer to have things separately in case of desasters.

So here I was, with a stack of pdf files - unsorted, and with quite a few non-articles crept in between them - and my database. Enter Qiqqa, a database/citation tool geared entirely towards .pdf collections. (If it were a little less geared towards those, and a bit more open and more import-friendly from EndNote's end, I would have considered switching to it completely.) In my process of checking out Qiqqa, I already tried to use it for sorting, organising, and EndNote-ing my pdf files, but it turned out to be a tiny bit less trivial than I had thought.

So I have made a clean slate in Qiqqa and have now tackled (again) the task of sorting my files and inputting them into EndNote. It's still a multi-step process, but much less tedious than before. Preparation step was to make three new folders for sorting: One to hold the batch of pdfs for processing, one to hold the exported files, and one for the "rejects" - files that have obscure bibliographical data that will have to be entered by hand.

Step one: Move a batch of pdf files from the big heap into the processing folder.
Step two: Import that folder into Qiqqa (or set it as watch folder). The programme will now index (and, if necessary, OCR) those files.
Step three: Use the inbuild BibTex-Sniffer to match bibliographical data to the individual files, and delete all the non-articles from the library.
Step four: Make sure to move all the files for hand processing from the processing folder to the "rejects" folder (else they will be lost), then delete them from the library.
Step five: Export bibliographical data to a .bib file.
Step six: Export complete library to the export folder.
Step seven: Convert BibTex-file to an Endnote .xml file using this nifty little programme.
Step eight: Import bibliographical data into Endnote (excluding duplicates). (I only had one minor glitch with importing up to now which seems to have been an incompatible record type number.)
Step nine: Add "pdf file available" or similar thing into a suitable field of each of the new references (this can be done quickly with "change and move fields").
Step ten: Move all the exported files from the "doc" subfolder in the export folder into the regular folder for referenced pdf files.
Step eleven: Delete everything from the export folder and the processing folder.
Step twelve: Delete all the entries of the library.

Then start over... until everything is processed. It takes some time, but on the other hand, it allows me to be sure I get everything referenced and lets me clear out all the other pdfs that crept in without too much woes. And with the possibility to do this in smaller batches, it's also not so overwhelming to add hundreds of BibTex entries at once.

(And if this blogpost has made you want a bibliography programme/database, here is a list of those currently available, including EN and Qiqqa.)

Thank goodness it's Friday! I'm really looking forward to the weekend - and one of the things I have planned to do is starting the Xmas baking. There shall be cookies!

Otherwise, my main item on the to-do list today is to finish the submission of a paper (another one about the spinning experiment). It's the deadline today... and yes, that proves that I am a perfectly normal person, submitting right on deadline day.

And once that is done, I am planning to clear a bit of the rubble, er chaos, off my work desk (which includes some updating of my book database). And maybe play a little with the small digital microscope that I now have...

And next week, it will be back to normal winter-type work.

Endnote is, according to their makers, the bestest citation software ever. As is Zotero (according to their makers). And probably a bunch of others as well.

Back when I was still trying out stuff and relatively new to the world of people working in physics (who have an affinity to LaTeX-the-programme), I wrote one article in TeX. I had to re-do it in MS Word to get it published, but this little stint meant that I got to know the powers of BibTeX, which is the citation system thingie that comes with TeX. It is, more or less, a reference database that you cite from, and the programme does the formatting.

Sound familiar? That's just what EndNote does with Word. Back when I then found out that there was, indeed, such a programme, I bought it for the hefty sum they asked for the student edition back then and have been using it ever since. There's connection files to get data from online library catalogues (something I should have used more, I now think, it might have saved me lots of work), style files that can be altered to suit individual needs, and both a field for "notes" (which I use for making general, personal notes about a reference) and "research notes" (which I use for typing excerpts or snippets including the page number so they can actually be used for working without the physical copy of the book). Plus there are custom fields - those I have used to mark whether I own a book (physical or digital copy), where it stands in my library (well, that's a work in progress to be honest), if I have pictures in my picture database (by noting down the prefix of the image files, such as schweppe_ which is, in the actual files, followed by a page number so I have files like schweppe_10.png) and if I have already read it or just jotted it down for future reading.

According to the version history, my purchase of the programme must have been in 2004. Which means that my version is a little... older. Now, I have no problem with older software (I'm happily using Word2000, and not planning to change from that), but sometimes, it pays to look for alternatives. Especially since my EndNote is sort of iffy on the connection files, with quite a few of them not working (or not working anymore), and I cannot download the whole set of new ones from the site (there is no such button in my installation menu).

Due to the recent mention of Mendeley by Phiala in the comments of some other post, I did a little looking and found Qiqqa - a .pdf organising software with built-in OCR, a wizard to help with filling out the reference data for each imported .pdf, and the possibility to cite to Word. I quite like it on the first try and am currently using it to get some order into my .pdfs. I have even considered changing to this programme from EndNote for all my referencing, but it does not do a few things that I have grown accustomed to, and I have also read that it's not too easy to modify citation styles (something I regularly need to do).

Qiqqa is intended for use over the web, with web storage of .pdfs, something which I don't need (and don't want). It allows "vanilla references", that is those without a .pdf attached, but then I get a nasty popup if I want to associate a file with it now. It also does import from EndNote - but not the "Research Notes". There is the possibility to search for duplicates, but I found it rather hard to compare the actual duplicates with each other and decide which one should stay and which one should go. I have not tried the citation thingie yet, but from the overall feel, it's very much geared towards .pdf files only, and I don't want to use two programmes. So at the moment, my plan is to use Qiqqa to get some order into my .pdf files, then export the data via BibTeX and a converter to EndNote, and merge the two databases. (That's the plan only, though - and it might change if problems with the import should arise. Or if Qiqqa should evolve some more, and then I might reconsider using it as my new reference database thingie.)

I'd be happy to hear about your experiences with Qiqqa, EndNote, Zotero or whatever you use!

I have been asked to consider turning my blog into a book, and I'm now pondering the idea. It's true that I have written a lot of stuff over these past years, but I'm not convinced enough of them would be non-internet-specific enough for a print version.

It is certainly less work than writing a book from scratch - but it's also sort of non-ordered, non-sequitur, and containing a lot of filler posts or link lists.

And I'm just not sure whether anyone would really want to buy a version of this blog in form of a book. So I would be grateful for any feedback - what do you think about blogs turned books? Are there any articles I wrote on that blog that you'd like to have in a print version? Or should everything rather stay as it was?

It is a very, really, utterly amazing thing how much work can go into a single book. There's this project I'm working on? Together with a friend half-way around the globe? We've spent oodles of hours, both, on the computer and chatting over how to fix this or handle that and on ourselves thinking and writing and editing and checking and noting.

And now we are nearing a stage where we can take a first look at what the thing has become. It has certainly changed a whole, huge lot from what it was (both in writing and conceptually) when we started. There's still a good bit of work to do, but things are getting clearer.

That's amazing. And a tiny bit scary - nearing a new stage.

Oh, on today's agenda, by the way (among a few other things): more work on that project. Which I will now be off to...

... again. After several months with very little to no rain, it seems as if nature is trying to catch up, and it has been pouring down now and again - with lots of heat and sometimes really humid air inbetween.

I don't really mind the sound of rain, though, and I'm happy that we do not have to water the lawn or the flowers and tomatoes outside. The cat, however, is not so pleased. She would much prefer staying dry when attending to necessities outside, and it took her a few minutes of sitting and looking into the rain before she decided she prefers to get a little wet from the outside instead of using the (perfectly dry and of course clean) litter tray inside. Cats. At least the birds don't mind the rain too much, so she can still have cat entertainment from the dry inside.

I have actually managed to do some proper writing yesterday for one of the articles I have to do, and I plan on doing some more today, so things are moving on. I feel a definite need for some holidays though - good thing that a summer break is not really far away now!

I have been writing this blog here since December 8, 2008. I am trying hard to post something interesting each day - or at least to post something, preferably interesting. In those three-and-a-half years, I have not had a single guest post. I actually did all the 728 posts myself. If I copy-paste any content into this blog, it's something like a call for papers that gets sent out with the aim of being distributed to interested persons - and I like to help in that.

Which means that to any person taking more than a very, very casual glance at my blog, it should be quite clear that I am not going to do post exchanges or post advertisements. Yes, I will plug my own stuff (hey, it's my blog, after all) and I will praise other things that I am happy about - but not because I was asked to do so, but because I want to. Because I am convinced that there is something in high quality for a fair price. And I have never gotten any compensation for my plugs. (Most of the "you are so going to want this" posts are about free internet resources anyways.)

So. I still get the occasional "oh your blog is sooo coool please visit my site" comment spam (which I delete) and, about as rarely, a mail praising my blog with a standard set phrase and then the offer of either a link exchange or "guest posts" coming from that other site, or some money if I advertise for a product in my texts. Usually, I just ignore them. Sometimes when I feel like it, I will take a look at the site, laugh, and then delete the email. But this morning I received one of the guest post offer mails that didn't only (as usual) make it very obvious the person writing it had not taken a good proper look at my blog (though becoming a "fan" right away)... that person didn't even know the real topic of the post I was offered as a guest post, which did not fit in at all with my own focus.

So I finally took this opportunity of total incredulity on my part (really? you really think I will now believe anything your site hosts is well-researched?) and sent that person an answer mail... raising the offer. With picture of Wil Wheaton collating papers. (If you have never heard about that and get weird offers sometimes too, do click the link. If you haven't heard about it and don't get weird offers... oh, click it anyways. At least I think it's funny.)

Hah. Something I had wanted to do for ages. Who says weird soliciting mails are not good for anything?