Micro-Science-Blogging

Here's a crazy idea, what if scientists posted results of their studies for the year, month, week, or hell, even their exciting result of the day, to YouTube? What if we blogged our science?

Let's say, for sake of argument, that I have an interesting result, not yet ready for publication in a journal but certainly worthy of the attention and maybe comments from other researchers. On the one hand, I could work on that idea further until it's ready for a "real" publication. That would probably involve some missteps, maybe some time lost (open the door for scooping), but will also make for a more complete product when it is actually released. 

On the other hand, I could open myself up for comments at a very early stage. Not only would the online "timestamp" validate my status as first to the result immediately, it would also give others a chance to comment, criticize, and potentially even initiate their own projects. When the project reaches a later stage I will have already incorporated "reviewer" comments into my manuscript, which will remain just as valuable as a formal report of the work. 

Perhaps such a system could work on a by-lab basis. Labs could become, in a weird way, their own publishing houses. Weekly updates on results could feed into the RSS feed of other labs. Just a thought

If anyone is wondering what regulation capture by industry interests looks like, this is it:

A Life in Energy and (Therefore) Politics (wsj)

The energy industry is mired with so many competing regulatory and industry interests that calling it an "energy market" is a complete misnomer. Note the intriguing comments made later on in the article about some "dirty: coal plants being used as bargaining chips by companies against state and local politicians who, having granted monopolies, are now beholden to those companies.

It's a real fine mess.

The Rational Optimist and Ricardo

I'm about halfway through The Rational Optimist and have to say that it's a bolstering case for the phenomenal power of markets and trade in human society. It's not exactly erudite, but it's a well versed look at the way trade and specialization have massively improved the lot of the average human in the past few million years and enabled us to grow to a population of several billion. Some of the points I like the most revolve around Ricardo's Law of Comparative Advantage, which the book describes as "the only result of social science that is both surprising and true". In honor of the recent free trade agreement between the US and several countries, I want to take a moment on this little law.

Briefly, the law states that if two individuals, nations, companies, whatever, have a different relative ability to produce a pair of different items, then one or both of the actors will be better off if they trade. So, for example, even if France is better at making both wine and cheese than England (probably true), both England and France will still benefit from trade if the relative cost of producing wine vs. cheese in the two countries are different.

What if France makes wine much more efficiently than cheese, and England does as well? It turns out, it doesn't matter. If the tradeoffs of wine vs. cheese are just different, that's all it takes for both countries to benefit.

What's marvelous about The Rational Optimist is that it couches these modern conceptions of economics in the ancient environment, before they were even realized. Let's say I make spears incredibly well, and I also run down and spear bison incredibly well (probably not true). I take a day to make a high quality spear, and a day to run down a bison. You, sadly, are terrible at both, taking 2 days to make a high quality spear, and 4 days to run down a bison.

Now we could both spend time in our own little spheres, making spears and running around all day. At the end of 6 days, you would have one spear, which you will have used bringing down a bison. I will have made 3 spears and brought down 3 bison.

After those 6 days, you have a brilliant idea. A brilliant idea that will keep you from running around all day and instead put my hands at work for you. Let's say that the next 2 days  look a lot like the 6 days before. I make a spear, and go skewer a bison with it. You're just be polishing off that first spear when I bring the bison home.

But now you throw your incredibly brilliant plan into effect. "Ugg!" you say, (we're cave man, that's the my moniker)  "I'll give you that spear for just 1/3 of your next bison!".

I think it through. 1/3 of my next bison? But I can have all the bison if I make the spear myself. But if I make my next hunting spear myself, I have to sit around all day making it, and won't be able to hunt until tomorrow. If I take this spear, I can hunt now. Let's see... 1 bison in 2 days, or 2/3rds of a bison in 1 day. I might not be able to figure out the fractions, but I know that I'm getting more. I take your spear, head out for the hunt, and that night I come back with my prize. You get 1/3 of a Bison, and never had to run around on the hot plains.

I won't belabor the point, but if you work out the math, over 6 days with the occasional spear trading hands, you still get your one bison by making 3 spears instead of 1 spear and one hunting trip. I am doing a lot more running, but I get 3 hunting trips without having to stop to make spears, and can make one hunting trip with my own spear and still have a day to spare. Calculated out, I get the same 3 bison, and a free vacation day!

Consider that this example is the most meager form of Ricardo's comparative advantage. Imagine that after a year of this arrangement, you get absurdly good at spear making. You've been doing it every day of the week after all, and you've had a chance to try some things out. Now, after figuring out how to use a rock to sharpen the tips and finding the best source of branches in the woods, you're turning out 2 or 3 a day! You're still only trading one for 1/3 of a buffalo, but now you can stock up on spears the first 2 days of the week and spend the rest of the time figuring out how to corral a buffalo calf and save it for later.

Clearly I've just sketched the barest bones of the argument here, and I highly recommend the book for a better treatment. The point, though, is that specialization and trade are massively powerful ways to improve our own lives and those of our trading partners. Imagine, in your own life, how difficult it would be to independently  create all the wonderful artifacts of human productivity you enjoy. What you get out of voluntary trade makes possible the life you lead today.

Minimum Publishable Unit

What the hell, I'll post again this year.

When is the right time to publish? Is it when you have an interesting result? Is it when your result is "complete", zipped up, lock-tight, slam-dunk? Is it when the result seems more likely true than not? Is it when you want to recruit others to the cause? Is it when you think it's good enough to impress people, and impress them in the amount that you need them to be impressed for your career to advance?

Sadly, I think the answer is often that last on my list. Publications are the the currency of science, and the only hard paper that demonstrates productivity on the public stage. Science is a big establishment, and our individual spheres are often tightly circumscribed, though they may seem vast from the inside. If I am applying for a job, and I have a Nature paper or a Cell paper (not happening any time soon, but let's just imagine for a moment), then I can put that on the desk and point to it. Say my competitor has a paper in PLoS Genetics. Still good, still the same paper, but different brand. I'd carry more weight with the Nature paper.

That brand seeking behavior is nothing new, and it's not actually the problem in itself. The problem is when the brand takes on a life of its own. If everyone buys Nike shoes because they are flashy and Nike, and not because the Nike brand speaks to quality, then where can you expect the quality to go? Similarly if Nature papers are sought because they are flashy, and hard to get, where does Nature go from there? Will Nature continue to speak to the quality of the underlying science?

My fundamental problem with the big name paper is that the results are often big and sweeping. They are rarely circumscribed, but elegant and well thought out. Imagine a demonstration that a single class of proteins perform some specific catalytic or signaling function. This in itself is worthy of being shared with the scientific community, as an informative work. But to get the Nature paper you need to show the protein is relevant in disease X, and that if you inhibit the catalytic activity you cure warts, cancer, and heart disease.

Of course inhibition of the protein won't turn out to actually cure warts, cancer and heart disease. Somewhere along the line a wrinkle will have been ignored. Some control, that in hindsight will be obvious, won't have been done. This is inevitable in the sort of expansive manuscripts that top journals demand.

I argue we should be more focused on rapid dissemination of our research, and broad feedback from the community early on in validating a result. I might run a smart, genome-wide screen, but you might have a better idea of how to interpret it. Someone else might recognize an important statistical error. Most importantly, someone else can *replicate* the experimental results early, before I truck on down the road with faulty assumptions.

The problem is not that the "minimum publishable unit" is too small, it's that it's too big, and that we focus on the papers and the journals and not on the results themselves. In the end, a Nature paper that's wrong is worth less than a small time journal article that's right. The Nature paper that's wrong may actually have negative value, having led labs down the wrong road. Next time you hear someone say that they have a Cell paper under their belt, ask next whether the result was replicated. Ask whether the result has, in fact, been important. Remember to value not the scientific articles, but the science itself.

Irritated in Chapel Hill

The scientific establishment is Democratic.

News to anyone? Probably not. After all, Democrats by and large promote increases in NIH and NSF funding, tout the importance of the scientific consensus, and so on. But when one is, oh, say, not a Democrat, and trying to do science within the field as it currently exists, there are bound to be frustrating moments.

Take, for example, two seminars in a year with national Democratic representatives. Let me add that these seminars are ambiguously mandatory, with a sign in sheet at many, though not all events, and that they are catered by the program. Imagine a room full of decidedly left leaning individuals with their representative on stage. These reps, so far as I can tell, have nothing to talk about but their political leanings and aspirations. And they'll have all the time in the world to tell you how important they think it is to support, Education and (National) Health Care. They'll also mention how they plan, at the Federal level, to fund Science, Education and (National) Health Care. Nevermind where the money is coming from...

Now I'm not saying no seminar has had a Independent, Republican, or otherwise non-democratically leaning individual who has weighed in, where asked, on political issues. It's just that I have yet to see such an individual given an open forum with a mandate to talk about politics. And perhaps I am being unfair, since I can't imagine any such speakers wanting to wade into the issues of a limited Federal role in that crowd. The Feds are our lifeblood, as it currently stands, and I am one of the few who bites the hand that feeds him. Hell, I have even applied for Federal grants, and benefit from State money given to our Cancer Center, so who am I to complain?

But of course that is exactly the ratchet of Federal authority, ever gaining constituents through patronage, that erodes limited government. I have to live within the system as it exists, even as I argue for its change, as I do here. I just wish that this group of ostensibly deeply thinking individuals were less overwhelmingly game to sign on with whoever holds the purse strings. And I wish they didn't strongly (though perhaps unintentionally) broadcast their inclinations from the top down.

I hate being a naysayer, and I hate biting back at a program that has given me a great deal. But in case people are suffering under the illusion that the MD/PhD program at UNC has built a scientific consensus that you should vote for David Price and Kay Hagen , I just want to make it clear that it has not.

The Hazard of High Throughput

We live in a high throughput age. Science is no exception. Microarrays, high throughput sequencing and spectrophotometers generate data on a scale and scope that would have taken years, decades, or centuries with the the old generation of technology. We are generating data on a scale that could not be conceived of a decade ago.

Great power... let's see what comes next....

The challenge, of course, is with the analysis. The data now generated is so massive that it cannot be simultaneously visualized or processed in its raw form. Nor can t-tests get you where you want to go with statistical analysis, so many are the comparisons being made.

Enter the world of the Bonferoni correction and the Benjamini-Hochberg false discovery rate. These statistical methods allow us to sift through such enormous data sets to focus on results that are significantly different from random expectation.

The hazard comes with these methods' complexity and somewhat obscure statistical assumptions. Many scientists are very well versed in the hypotheses of their discipline, but less so on the mathematics. There are so many ways to go wrong in applying these methods in a cookie cutter way that it boggles the mind. Along the lines of "Correlation implies causation" there are other such gems as: "Difference in significance does not imply significant difference".

This last mistake was featured prominently in stage 1 of a statistical analysis in a manuscript from a good lab that just passed my boss' desk. This lab has produced prominent publications in the past, and I was surprised to see this in their analysis.

What most surprised me was that the statistical method, buried deep within the methods section at the end of the manuscript, did not arouse the ire of my boss or the other lab member who read the paper. It was viewed as a good enough an answer to a tough problem. That, plus the prominence of the last author, led to a minor note somewhere in the review.

Reviews don't have dissenting opinions, but let me put one here. Statistical methods are important. So important that, in a paper that uses a high throughput method at its start, they often form the backbone for every follow up experiment. They should not be relegated to a footnote in the back and they should, wherever possible, be declared before the data are even generated, to avoid the nefarious problems of overfitting.

Papers that use statistic need statistically minded reviewers. If we aren't careful, we'll be fooled by randomness.

Formulary of Ten

Here's a quick question: if you had to trim the formulary of a hospital down to just 10 adult meds, what would they be? I'm sure a careful and informed individual could make a comprehensive choice based on "Quality of Life Year" information and some epidemiology of the area. Here's what I am thinking:

1) Morphine
2) Aspirin
3) One or two antibiotics (Guess who doesn't remember micro enough to know which ones? I am guessing Cipro/levo and Vancomycin)
5) Insulin
6) Beta-blocker
7) A statin
8) Doxorubicin (What my boss called: the one chemo you'd bring to a desert island)
9) Warfarin
10) Lidocain

Thoughts? It might be a little over-covered for heart disease and under-covered for diabetes. Also, is it even worth it to have a chemo agent, from a utilitarian perspective? I know that's heretical from someone who is interested in cancer research, but how much life are you really adding with one therapy alone?

Also: does Morphine/lidocain deserve a place? Pain management is important, and I didn't want to leave it out. But when there are other diseases being left uncovered, should this be a top priority?

Anyone care to offer an opinion? It's an interesting way of thinking about priorities in U.S. medicine, and pondering how much we take for granted. What medicines are those that we can't make due without?

Peer Review

If you've spent some time in graduate school you might have learned a thing or two about the open secret about peer review: it's not entirely done by peers and it's not always that thorough of a review.

Like it or not, grad students often find these reviews on their desk. I've heard of PIs leaving the job to grad students alone, though this does not occur in my lab. It's understandable, though, that this might happen. PIs have huge demands on their time. As invaluable as their opinions are, there is no way some PIs can field every manuscript. Apparently they have their own concerns about peer review (some of which apparently involves colorful dinos)

But what of the reviews performed by graduate students? There are two ways of looking at the duty of peer review.

Looked at one way, a peer review is a learning opportunity. Often the material details development on the edge of some field that contacts at least tangentially upon the student's research. It's a chance to see another lab's raw, early draft manuscript, and learn what merits publication in high level journals and what does not. It is a learning experience, and a chance to contribute to the body of science as a whole.

But let's just step out of the shiny world of gumdrops and candy canes for a moment. Peer review can be an inane chore. While students provide some value added to the journal and the author of the manuscript, it's harder to see where the review process benefits their progression in the ladder. Their role in the review is, effectively, anonymous, and comes with no honor, distinctions, gold stars, pats on the head, brownie points or first author publications.

Stated simply, there is no clear match between incentives and the quality of the peer review. Mistakes are often buried somewhere deep in the unending, jargon-filled paragraphs of the methods section. Whether a grad student takes the effort to check these methods, line by line, comes down to a question of how many hours (if any) of sleep they might prefer to have that evening.

Where's the incentive to dig in? Some grad students seem to possess a deep personal drive to throw other scientists under the bus, but it's probably a minority at most institutions. We can't rely on pure sadism to drive the scientific engine. There must be a way to reward careful and well considered reviews, particularly where they find obscure errors and tenuous methods.

I don't claim to know what the reward should be or how it could be structured. I've considered the possibility that confidential peer review is a mistake, and that publications should instead be edited by the journal seeking to publish. If Nature wants the value added of an expert opinion in the field, let them pay for it! Certainly they demand payment for their subscriptions, so why should their product be provided for free?

At risk of shifting to a seemingly radical alternative, perhaps open access and open comment system is the way to go. Take all comers that pass a basic editorial spot check, and allow insightful, observational comments to come from the community. Those comments can then be tied to the reputation of those who make them. Great insights can be noticed, and unnecessary bickering (I'm looking at you, reviewer #3), can be ignored. Online systems for grading and sorting comments based on reputation systems exist in many forms. Perhaps its time to turn them loose on science.

Concurrently, give labs the hard task of determining their own publication threshold. Perhaps more self review will go on in-house if authors know that they can publish whatever they please, and that they'd better get it right the first time. Perhaps some systems could allow papers to have a 'versioning' system, wherein they could be updated (to a point) to reflect public comment.

I don't claim that pay-to-review or open-comment systems solve the problems inherent in the current publication regime, but I think they deserve consideration. Let us at least recognize that good work deserves good incentives, and that the adding motivation to peer-review can only improve our scientific rigor.