The revolution in immunotherapy is incredibly exciting. Cancers are being treated, and occasionally, apparently, cured, by tweaking a patient's own immune system. It is a thrilling new development, and a hope to millions of people.
But we need to show some sobriety.
To understand why we haven't found "the cure for cancer", we need to understand why immunotherapy has worked so well in the places it has been tried.
The first is CART therapy. The CART acronym stands for "Chimeric Antigen Receptor T-cell". This method is completely brilliant, in that it uses the killing power of cells that your body already has in abundance, T-cells, and adds just a few key pieces. A T-cell, as it turns out, fights diseases by identifying cells that look infected, and mercilessly killing them. It can do that because the virus is often sitting around on the surface of the cell, or the cell "presents" little bits of the virus that the T-cell can identify are foreign.
A cancer cell, though, often doesn't have bits of virus to present. It just has the usual proteins of every other cell in the body, with a few very small exceptions. T-cells are carefully trained, in a process called "negative selection" in the thymus, to avoid attacking the body's usual proteins. Imagine the consequence if this didn't happen: you'd have your T-cells trying to kill you every minute of every day. Cancer starts off as being human, and then accumulates mutations that make it corrupt. But that starting point, a normal human cell, is invisible to your immune system (as it should be).
However even normal human proteins can occasionally be useful targets. Some cancers, particularly certain cancers of the blood, have very specific proteins that differ from the rest of the body. These are called CD19 and CD20. These proteins basically appear in only two places: the cancer and the normal white blood cells that gave rise to the cancer. Since the immune system is usually completely out of whack anyway, killing off all those few normal white blood cells along with the cancer is acceptable collateral damage. If only they had a way to target those proteins...
Then comes the "one weird trick". Thanks to a whole lot of people, people like Carl June at UPenn, a plot was hatched to give the T-cell a specially tailored gene to make it attack those white blood cell proteins instead of viral proteins. The gene could look a lot like the genes a T-cell already has, but with some special changes slapped together to make that T-cell absolutely ravenous for the sweet taste of CD19 or CD20. When they see their prey they turn into killing machines, and they multiply to kill even more.
It turns out this totally works! Just this year the initial approvals for treatment of childhood Acute Lymphoblastic Leukemia (one of those blood cancers I mentioned), got approval by the FDA to be delivered to patients as a potentially life-saving cure. It's a tremendous accomplishment, and is just the beginning of a flood gate of new therapies. It's a very exciting time to be working on these methods, as we explore how to bring other cancers into the target sights of these CART therapies.
Unfortunately, there is a catch. For one, because the treatment is so expertly honed on a specific protein, the cancer has quite a few ways to dodge the attack. For one, it can just stop making the protein. It turns out that's not so hard for a cancer cell to do, since it's often busy accumulating a rather tremendous number mutations over time, and some cancer cell will get lucky and hit, for example, CD19. Having erased the target on its back, the cell can go on multiplying unencumbered by the astonishingly intricate therapy we have devised.
But additionally, it turns out that ALL is sort of rare among tumors for having such a uniquely defining protein just humming around on the cell surface waiting to be targeted. While cancer cells are certainly very different from non-cancer cells, in many cases a patient's individual tumor may be entirely different from another patient's individual tumor even if they have the same kind of cancer. It's an unfortunate state of affairs that means that finding a target that specifically identifies a patient's cancer, and not all the bystander normal cells that make us who we are, one might have to resort to a specific cure for a specific patient.
A truly "personalized medicine" is a tall order. We might have to develop a vast arsenal of weapons, only to pull out a single, specialized stake to drive into the heart of a given patient's disease.
The immune system, however, is dazzlingly complex, but I wonder if it intrinsically contains the key to the puzzle. The immune system does something very specific in response to an infection that I find staggering: it undergoes evolution. Your body, in service of your survival as an organism, actively uses evolution to identify a receptor that matches a given enemy in the field. To oversimplify the system enormously I'll explain as follows.
When a virus attacks, the immune system throws a bunch of cards in the air and lets them come down scattershot. Those that land closest to a match for the invader start to tickle the cell into reproducing, activating a program that increases the cell's tendency to mutate as well as create new copies. Effectively, this means that the cards that landed nearest to the virus in the space of possible matches send out their own little scattershots again, this time in a tighter area, trying to move closer to a perfect match for their enemy. This process continues until a good match is formed, and the cells are effective enough at their job to eliminate the enemy.
As I mentioned at the top, though, a cancer cell looks a whole lot like a human cell. And there are specific systems designed to make sure that, even as it's evolving towards a better weapon to target, for example, a virus, anything that matches too well with a normal human cell is treated as off-limits. That is great, usually, because it keeps us from developing autoimmune diseases. But if I have a tumor, I desperately want a very certain kind of autoimmune disease, an anti-cancer kind.
We actually have the tools to remove some of those "off-limits" signs. A whole other type of immunotherapy, called "checkpoint inhibitors" allow the body to wander into those danger areas and explore the space of autoimmune disease. That's fantastic if it ends up giving you a cancer targeting T-cell, but often there are complications where a set of T-cells go after the wrong target, like the normal skin, or the gut, or the liver. That can be exceedingly bad news, and it puts a limit on just how far we can go with those therapies.
The benefit, though, is that these checkpoint inhibitors allow the immune system to devise its own attack plan. Within the patient, a process of evolution occurs to counter the tumor's own evolving approach. That is a fantastically powerful process. No pill cooked up in a lab somewhere can shift and feint with the tumor, and that fact has too frequently doomed some of our recent targeted therapies.
I wonder, though, if we can somehow get the best of both worlds. The incredible power of building a T-cell from scratch, in the lab, combined with the power of evolution inherent in the immune system. I can imagine a world where, in the contained safety of a dish, a host of a patient's own T-cells are modified and reacted against the patient's tumor to develop that "personalized" book of armaments. Those could then be vetted not in the living chaos of the patient's own body, but again in a dish, to weed out dangerous autoimmune functions. Thus processed, if these cells could then be expanded and put back into the blood to attack a tumor, they would be a hailstorm of guided missiles.
This might be a fantasy, but there's some beauty in it. Attack somatic evolution with somatic evolution. If only Darwin could see us now.
Thursday, December 7, 2017
Saturday, July 9, 2016
PGY2
Well, that was a whirlwind.
The first year of residency is known to be a gauntlet. I thought I had dodged the bullet by picking a pathology residency instead of internal medicine. Nope! Sure, I bet they have it worse, but I have still experienced the absurd working hours and learning cliff of a starting resident. I have still been humiliated by my ignorance and deathly afraid of hurting patients with my incompetence. I've worked hard, burned out, burned back on, and struggled to find equilibrium.
And maybe, now, I've sort of found it? After a year, I don't think my inexperience has really harmed a patient. I've had layers of protection, from the Pathology Assistants (PAs) at the grossing bench, to the attending physicians at sign out. I've kept my wits about me and recognized my errors, and corrected them. I've had my senior residents, who are the perhaps the most understanding what what we are going through, close at hand with encouragement and wisdom. Perhaps most importantly, my lovely wife has nourished me, both with good cooking and emotional support. No one gets through this without help.
To be sure, the hours were unreasonably brutal. They remain unreasonably brutal, and will continue to be unreasonably brutal. Part of this is an artifact of my training path: college, 2 years in a lab, then an 8 year MD/PhD program. I am older than most used to be at this phase of training, but no more experienced in the specific area of practice. I have a lot of OTHER experiences that inform and educate my ability to do my job. I know some advanced math, computer programming, genomics and lab skills that could pay off someday. Right now it's all the time honored aspects of anatomy and histology, and I sure didn't come in knowing a whole lot of either of those.
(As a parenthetical: I continue to maintain that the way physicians are trained in this country is absurd. My friend from Ireland who had the benefit of entering medicine out of high-school, as is standard in much of Europe, and is arguably one of the most capable residents in the program. 4 year college before medical school is an extremely expensive luxury that might be totally pointless.)
I also have a wonderful family that demands time and attention. It's painful every morning to leave my son, and even more painful on days when I come home to find him already asleep. There's a different kind of pain reserved for when he wakes up at 4:30 AM on days that I have worked past midnight. I sometimes envy the younger residents who can devote themselves more fully to learning the craft.
But, as the new first years arrive, I realize how much has been learned while running this gauntlet. Sure, I could have learned more, but in one year I have gone from understanding extremely little about the discipline to knowing at least an outline of every area of pathology. Sure, I wouldn't in a million years trust myself with signing out cases at this point, but I now see the map of the territory I need to explore, and come to know. Its boundaries are mostly drawn, and some of the major features in the terrain are already sketched in.
In short, I am hopeful. Not that I will be working any less hard, but that my work is bearing, and will continue to bear fruit. I am learning more and more HOW to learn, and that ratchet will carry me up the cliff in safety. I look forward to the view from the top.
The first year of residency is known to be a gauntlet. I thought I had dodged the bullet by picking a pathology residency instead of internal medicine. Nope! Sure, I bet they have it worse, but I have still experienced the absurd working hours and learning cliff of a starting resident. I have still been humiliated by my ignorance and deathly afraid of hurting patients with my incompetence. I've worked hard, burned out, burned back on, and struggled to find equilibrium.
And maybe, now, I've sort of found it? After a year, I don't think my inexperience has really harmed a patient. I've had layers of protection, from the Pathology Assistants (PAs) at the grossing bench, to the attending physicians at sign out. I've kept my wits about me and recognized my errors, and corrected them. I've had my senior residents, who are the perhaps the most understanding what what we are going through, close at hand with encouragement and wisdom. Perhaps most importantly, my lovely wife has nourished me, both with good cooking and emotional support. No one gets through this without help.
To be sure, the hours were unreasonably brutal. They remain unreasonably brutal, and will continue to be unreasonably brutal. Part of this is an artifact of my training path: college, 2 years in a lab, then an 8 year MD/PhD program. I am older than most used to be at this phase of training, but no more experienced in the specific area of practice. I have a lot of OTHER experiences that inform and educate my ability to do my job. I know some advanced math, computer programming, genomics and lab skills that could pay off someday. Right now it's all the time honored aspects of anatomy and histology, and I sure didn't come in knowing a whole lot of either of those.
(As a parenthetical: I continue to maintain that the way physicians are trained in this country is absurd. My friend from Ireland who had the benefit of entering medicine out of high-school, as is standard in much of Europe, and is arguably one of the most capable residents in the program. 4 year college before medical school is an extremely expensive luxury that might be totally pointless.)
I also have a wonderful family that demands time and attention. It's painful every morning to leave my son, and even more painful on days when I come home to find him already asleep. There's a different kind of pain reserved for when he wakes up at 4:30 AM on days that I have worked past midnight. I sometimes envy the younger residents who can devote themselves more fully to learning the craft.
But, as the new first years arrive, I realize how much has been learned while running this gauntlet. Sure, I could have learned more, but in one year I have gone from understanding extremely little about the discipline to knowing at least an outline of every area of pathology. Sure, I wouldn't in a million years trust myself with signing out cases at this point, but I now see the map of the territory I need to explore, and come to know. Its boundaries are mostly drawn, and some of the major features in the terrain are already sketched in.
In short, I am hopeful. Not that I will be working any less hard, but that my work is bearing, and will continue to bear fruit. I am learning more and more HOW to learn, and that ratchet will carry me up the cliff in safety. I look forward to the view from the top.
An Ode to the Electronic Medical Record
Dearest electronic medical record,
Thank you for permitting me to read all the notes for my patients, going back for years, if not decades, in a typed, readable font. Thank you for storing all the lab values, also going back for decades, with a clearly available range of normal values for the laboratory that generated those values. Imaging results, from PET scans to chest X-rays, are similarly available at the click of a button. And thanks, also, for helping with billing. I don't really know what you're doing with all that, but I understand the the hospital administrators really like particular feature. It's really hard to underestimate how important these three things have been in improving care in hospital settings.
I just want to encourage you, electronic medical record, going forward, to learn from your mistakes and maybe offer some constructive criticisms.
You're starting to get a little big around the midsection. I call this "Note bloat". Doctors today are very busy, serving more and more complex patients, and often at a fairly high level of specialization. That means its more important than ever that they be able to communicate succinctly. But when I try to use you, sometimes I have to scroll through pages of auto-generated ... I hate to say it ... garbage, before I get to a decently written paragraph written by a human who has seen the patient recently. That paragraph can be located just about anywhere in the text, and is sometimes, bizarrely, not anywhere to be found.
You're also slowing down a bit. Maybe because of the bloat, you have some trouble with load times. Google is great in no small part because it is so damn fast. You go to a simple page that loads quickly, type a search, and get relevant results from the entire internet quickly. Medical records are big, sure, but why on earth does it take orders of magnitude more time to search the medical record than it takes to search the entire internet? Time is money, but it's also user-sanity. I hear that there are methods like indexing which can improve this kind of thing.
On a similar note, you're loosing agility. Record systems like you used to be bespoke amalgamations that individual hospitals and practices assembled from parts scavenged from medium size software companies and their own in-house ingenuity. Today, it's pretty much EPIC. While EPIC is often customized at time of purchase, the resulting software is hard to change.
Here's an example of the above points. I often work in a frozen section lab. We offer rapid tissue diagnoses, mostly telling surgeons whether a specimen they send is cancer vs. not cancer, in 20 minutes or less, while the patient is on the operating table. It's critical that we know which patients are coming in advance, and be able to look up their history. Looking up the histories the night before, I read notes on dozens of patients, often in the realm of about 30-40 or so. I get that list by culling a list of hundreds that are planned for the OR that day. I need to efficiently load each patient's record, find the notes that are relevant to the operation being performed, identify key elements of the imaging history and past medical history, and possibly identify prior operations that may alter the anatomy of the specimen we are going to process. This process can take hours, but of course it is only possible because we have an electronic medical record in the first place.
In the first step, culling a list of hundreds of operations down to those that actually are likely to require a frozen section diagnosis, currently makes use of custom computer programs written in the pathology department. Ideally, these programs could be integrated into a specialized "pathology view" of the OR schedule.
The second step, loading the patient record, requires 5 clicks, each of which can have a slow load time. There is also an issue where the program brings up multiple patient records from what should be a patient unique medical record number. This introduces a possibility of error (it has not, yet, but it shouldn't even be possible). Ideally the entire pre-filtered list of patients in the OR that day could be preloaded with one click, allowing access to each patient quickly and unambiguously.
The third step, reading through the records, requires me to identify important notes for the particular surgery of the day from a half dozen to hundreds of other recent notes. Many doctor's notes aren't that important for my specific purpose, e.g. the patient's physical therapy notes. Others are extremely important, such as the note from the surgeon seeing the patient before the operation. There is no way to generate customized algorithms to search, filter, and sort these notes based on what the operation is, who is doing it, and what sorts of notes we (as pathologists) generally find most useful. Sure, there is a search function, and a filter function, but these are clunky, slow, and very limited in their customization options. Better than nothing, but less than ideal.
The fourth step, actually reading those notes, frequently runs into the problem of note bloat. Thank goodness I don't have to read doctors' individual handwriting, but I do have to scroll through a lot of auto-generated content. That content is often there, sad to say, for the purposes of billing. Sometimes you get to the end of a note and realize that the entire thing is a billing document. This takes minutes, per patient, which adds up to hours over the course of a week. I can never get that time back, and I'm not really learning anything but what font sizes and formats the auto-generated stuff normally uses so I can scroll quicker.
We can do better! It's not even that complicated! Notes are really just formatted text stored in databases. A physician with a little programming know-how could write very simple programs that would save hours per week for doctors in a their specialty. But there is no avenue to perform such customization. Moreover, as the marketplace becomes increasingly dominated by very few corporate providers, if not a single dominant provider, will there be any motivation to improve the product?
Electronic medical record, you have so much potential, yet untapped! I dearly hope that you see it. If not, perhaps you could attend some motivational seminars. Get some help from some computer literate medical colleagues! With a little ingenuity, I think you can shed the weigh, speed up, and improve your agility. It'd be great for your health, ours, and our patients,
Sincerely,
A Faithful User
Thank you for permitting me to read all the notes for my patients, going back for years, if not decades, in a typed, readable font. Thank you for storing all the lab values, also going back for decades, with a clearly available range of normal values for the laboratory that generated those values. Imaging results, from PET scans to chest X-rays, are similarly available at the click of a button. And thanks, also, for helping with billing. I don't really know what you're doing with all that, but I understand the the hospital administrators really like particular feature. It's really hard to underestimate how important these three things have been in improving care in hospital settings.
I just want to encourage you, electronic medical record, going forward, to learn from your mistakes and maybe offer some constructive criticisms.
You're starting to get a little big around the midsection. I call this "Note bloat". Doctors today are very busy, serving more and more complex patients, and often at a fairly high level of specialization. That means its more important than ever that they be able to communicate succinctly. But when I try to use you, sometimes I have to scroll through pages of auto-generated ... I hate to say it ... garbage, before I get to a decently written paragraph written by a human who has seen the patient recently. That paragraph can be located just about anywhere in the text, and is sometimes, bizarrely, not anywhere to be found.
You're also slowing down a bit. Maybe because of the bloat, you have some trouble with load times. Google is great in no small part because it is so damn fast. You go to a simple page that loads quickly, type a search, and get relevant results from the entire internet quickly. Medical records are big, sure, but why on earth does it take orders of magnitude more time to search the medical record than it takes to search the entire internet? Time is money, but it's also user-sanity. I hear that there are methods like indexing which can improve this kind of thing.
On a similar note, you're loosing agility. Record systems like you used to be bespoke amalgamations that individual hospitals and practices assembled from parts scavenged from medium size software companies and their own in-house ingenuity. Today, it's pretty much EPIC. While EPIC is often customized at time of purchase, the resulting software is hard to change.
Here's an example of the above points. I often work in a frozen section lab. We offer rapid tissue diagnoses, mostly telling surgeons whether a specimen they send is cancer vs. not cancer, in 20 minutes or less, while the patient is on the operating table. It's critical that we know which patients are coming in advance, and be able to look up their history. Looking up the histories the night before, I read notes on dozens of patients, often in the realm of about 30-40 or so. I get that list by culling a list of hundreds that are planned for the OR that day. I need to efficiently load each patient's record, find the notes that are relevant to the operation being performed, identify key elements of the imaging history and past medical history, and possibly identify prior operations that may alter the anatomy of the specimen we are going to process. This process can take hours, but of course it is only possible because we have an electronic medical record in the first place.
In the first step, culling a list of hundreds of operations down to those that actually are likely to require a frozen section diagnosis, currently makes use of custom computer programs written in the pathology department. Ideally, these programs could be integrated into a specialized "pathology view" of the OR schedule.
The second step, loading the patient record, requires 5 clicks, each of which can have a slow load time. There is also an issue where the program brings up multiple patient records from what should be a patient unique medical record number. This introduces a possibility of error (it has not, yet, but it shouldn't even be possible). Ideally the entire pre-filtered list of patients in the OR that day could be preloaded with one click, allowing access to each patient quickly and unambiguously.
The third step, reading through the records, requires me to identify important notes for the particular surgery of the day from a half dozen to hundreds of other recent notes. Many doctor's notes aren't that important for my specific purpose, e.g. the patient's physical therapy notes. Others are extremely important, such as the note from the surgeon seeing the patient before the operation. There is no way to generate customized algorithms to search, filter, and sort these notes based on what the operation is, who is doing it, and what sorts of notes we (as pathologists) generally find most useful. Sure, there is a search function, and a filter function, but these are clunky, slow, and very limited in their customization options. Better than nothing, but less than ideal.
The fourth step, actually reading those notes, frequently runs into the problem of note bloat. Thank goodness I don't have to read doctors' individual handwriting, but I do have to scroll through a lot of auto-generated content. That content is often there, sad to say, for the purposes of billing. Sometimes you get to the end of a note and realize that the entire thing is a billing document. This takes minutes, per patient, which adds up to hours over the course of a week. I can never get that time back, and I'm not really learning anything but what font sizes and formats the auto-generated stuff normally uses so I can scroll quicker.
We can do better! It's not even that complicated! Notes are really just formatted text stored in databases. A physician with a little programming know-how could write very simple programs that would save hours per week for doctors in a their specialty. But there is no avenue to perform such customization. Moreover, as the marketplace becomes increasingly dominated by very few corporate providers, if not a single dominant provider, will there be any motivation to improve the product?
Electronic medical record, you have so much potential, yet untapped! I dearly hope that you see it. If not, perhaps you could attend some motivational seminars. Get some help from some computer literate medical colleagues! With a little ingenuity, I think you can shed the weigh, speed up, and improve your agility. It'd be great for your health, ours, and our patients,
Sincerely,
A Faithful User
Saturday, July 18, 2015
The Learning Cliff
I've heard the beginning of residency called a steep learning curve. I am finding that it feels, at times, more like a learning cliff. You slam yourself against it repeatedly and attempt to find some handholds on your way up, while girding yourself for some inevitable slides back down.
I can only hope that it is not a Sisyphean adventure. I am told that it is not. It is, of course, partly just the usual "learning the ropes" that comes with a new job. On top of that it is confronting the vast bounds of ignorance that you previously knew you suffered from, but successfully were able to ignore. On top of THAT it is navigating the social dynamics intrinsic to any tightly knit group of coworkers who have to learn to work together for 80 hours a week (plus all that time hitting the books trying to make sense of what just happened).
I did manage to have an interesting conversation with an attending about just that latter topic, the 80 hour work week. Of course there are still those who decry its limitations, and I would say that I used to be one of them. However now that I am confronting the realities of both that work week and trying to raise a 3 month old, I much better understand the reasons for the changes. It is simply impossible to lead a balanced life in the modern age, take care of all the other aspects of life, and work much greater than 80 hours a week unless you are the sort (and more power to this sort) that doesn't need to sleep very much at night. For those who have married their job, instead of other people, perhaps more can be asked. But for those of us with families, it would be too much. I think it would impart a kind of spiritual sickness on the experience of residency for those like myself that would be devastating for the profession as a whole.
There is a fantastic piece in the New York Review of Books that describes some of the changes that led to the modern residency. To be sure, these comments apply as much to internal medicine as to my own discipline, pathology. The academic hospital environment has been compromised by the economic requirements of what is, fundamentally, a large corporation. It should be required reading for anyone entering residency (really, anyone entering medical school).
That said, I am still somewhat of an advocate for Malcom Gladwell's 10,000 hours to becoming an expert mantra. 160 hours down, 9,840 to go.
I can only hope that it is not a Sisyphean adventure. I am told that it is not. It is, of course, partly just the usual "learning the ropes" that comes with a new job. On top of that it is confronting the vast bounds of ignorance that you previously knew you suffered from, but successfully were able to ignore. On top of THAT it is navigating the social dynamics intrinsic to any tightly knit group of coworkers who have to learn to work together for 80 hours a week (plus all that time hitting the books trying to make sense of what just happened).
I did manage to have an interesting conversation with an attending about just that latter topic, the 80 hour work week. Of course there are still those who decry its limitations, and I would say that I used to be one of them. However now that I am confronting the realities of both that work week and trying to raise a 3 month old, I much better understand the reasons for the changes. It is simply impossible to lead a balanced life in the modern age, take care of all the other aspects of life, and work much greater than 80 hours a week unless you are the sort (and more power to this sort) that doesn't need to sleep very much at night. For those who have married their job, instead of other people, perhaps more can be asked. But for those of us with families, it would be too much. I think it would impart a kind of spiritual sickness on the experience of residency for those like myself that would be devastating for the profession as a whole.
There is a fantastic piece in the New York Review of Books that describes some of the changes that led to the modern residency. To be sure, these comments apply as much to internal medicine as to my own discipline, pathology. The academic hospital environment has been compromised by the economic requirements of what is, fundamentally, a large corporation. It should be required reading for anyone entering residency (really, anyone entering medical school).
That said, I am still somewhat of an advocate for Malcom Gladwell's 10,000 hours to becoming an expert mantra. 160 hours down, 9,840 to go.
Tuesday, March 10, 2015
Fourth Year Blues
I have been told by many that I should savor my 4th year medical school year. I have had a few months off already, and now have three in a row coming up. I will, of course, be using it for research. Others travel abroad. Some work on volunteer projects. A choice few will fill that time with coursework.
But this 4th year Spring is, in large part, deadspace. A big gap in the rush towards residency. I realized a few days ago that, at this point, I have fulfilled all the requirements that will make me a doctor. I have taken all of the tests, sat in all of the classes, spent all the time on the wards. There is a one week course at the very end that I need to take, but it is largely a formality.
And yet, I do not get my degree until May. Until that time I am just like any medical student in the hospital, 3rd or 4th year. I cannot write orders, I cannot write notes that count for billing purposes, and I cannot do any work outside the hospital as an M.D., patient care or no. While I have all the freedom in the world, I also have little real responsibility and no opportunity to up my game before being slammed by residency.
This seems foolish. Why would such an expensive system waste such valuable time? In all honesty, I can't figure out the reason. Perhaps it's that the home institution, with its students already matched with residencies, has little incentive to continue to invest time and effort in them. Perhaps more likely, the fact that we are not formally MD's makes us a legal liability and therefore we cannot be given intern level capabilities. Equally possible, perhaps this is what generations of students have demanded: an eye in the storm.
Regardless of the reason, it doesn't have to continue. Waiting for a degree prevents students from taking the next steps in their education when they are ready. It makes the hospital less efficient. It limits an educational opportunity. Residency can start in June, that's fine. We can still walk across the stage in May. In the interim, though, let's recognize the progress we have made for what it is and grant the degree when it has been earned, be that in April, March, or even January! The potential uses of the ensuing time would likely be as varied as they are today, perhaps more so, as some former students might use their new credential in creative ways. Let's unleash this trapped potential.
But this 4th year Spring is, in large part, deadspace. A big gap in the rush towards residency. I realized a few days ago that, at this point, I have fulfilled all the requirements that will make me a doctor. I have taken all of the tests, sat in all of the classes, spent all the time on the wards. There is a one week course at the very end that I need to take, but it is largely a formality.
And yet, I do not get my degree until May. Until that time I am just like any medical student in the hospital, 3rd or 4th year. I cannot write orders, I cannot write notes that count for billing purposes, and I cannot do any work outside the hospital as an M.D., patient care or no. While I have all the freedom in the world, I also have little real responsibility and no opportunity to up my game before being slammed by residency.
This seems foolish. Why would such an expensive system waste such valuable time? In all honesty, I can't figure out the reason. Perhaps it's that the home institution, with its students already matched with residencies, has little incentive to continue to invest time and effort in them. Perhaps more likely, the fact that we are not formally MD's makes us a legal liability and therefore we cannot be given intern level capabilities. Equally possible, perhaps this is what generations of students have demanded: an eye in the storm.
Regardless of the reason, it doesn't have to continue. Waiting for a degree prevents students from taking the next steps in their education when they are ready. It makes the hospital less efficient. It limits an educational opportunity. Residency can start in June, that's fine. We can still walk across the stage in May. In the interim, though, let's recognize the progress we have made for what it is and grant the degree when it has been earned, be that in April, March, or even January! The potential uses of the ensuing time would likely be as varied as they are today, perhaps more so, as some former students might use their new credential in creative ways. Let's unleash this trapped potential.
Tuesday, February 24, 2015
Yield
I managed to somehow end up at a roundtable discussion recently on the topic of funding reform and the future of science. Of course there was some of the usual clamor for more funding, as there has been and always will be. There was also some productive discussion of reforming scientific funding in the biomedical sciences, particularly scientific training.
Many opinions were aired, but there were a few insights that I thought resounded most clearly above the hubbub of what is, of course, a very political question.
1) Cull the herd: Do not allow students to continue Ph.D. degrees if they are unlikely to be successful. There was one individual who offered a clear comparison between what is done in the biomedical sciences and what is done in economics. The benefit of this are, at the very least, less delusion from students thinking that they might be able to go on in academia and less crush for grants when Ph.D.'s go on to apply for faculty positions. The cost is less cheap labor available to PIs in the biomedical sciences.
2) Numerator vs. Denominator: How do we avoid the simple problem of increasing Ph.D.s and constant (or decreasing!) funding. Are there operational efficiencies that can be achieved or some kind of restructuring of granting mechanisms that allow for continued scientific output, career stability, and stable costs? Interestingly the fundamental nature of this question was often overtaken by discussions of how nice individual programs might be in terms of some elements of this equation (e.g. a career track for core lab administrators might permit both continued scientific output and career stability... but at what cost?)
What is the cost, what is the return? In the end, this is an economic question. We in science are at risk of confusing the return to ourselves as the return to the investor. As academics, perhaps this is forgivable. We are, after all, obsessed with the deep questions of science. How does the cell work? How does the genome lock in a frozen, endlessly replicated organic molecule, the blueprint of life? How does cancer evolve? How can we better treat disease? What is good for us in our quest must be good for society, should we successfully answer these questions.
But the fact of the matter is much more complex. Graduate school is not a zero sum game, nor is economics. But the allocation of federal funding, sadly, is. The questions above are never fully answered, they are only answered in part. An attention to yield is paramount. As should be obvious, the yield equation is not the sum of impact factors of all the papers written by a set of investigators. It is the sum of the benefits to the individuals of the nation (the world?) over the centuries.
Whether or not we have stable careers is irrelevant latter sum of benefits, except inasmuch as stable careers allow researchers to do fundamental work (I think that they do, but to less extent than we might claim). What is more relevant, as far as federal funding is concerned, is the fact that a Ph.D. that is underutilized may be a wasted investment. In addition to the costs of funding the Ph.D., society has lost the other work that that presumably intelligent and capable individual could have achieved in that time in another line of work. Depending on the person, this may represent hundreds of thousands of dollars of loss. If they by exiting the academic track in fact exit the job market completely, the loss may even reach the millions.
The fact is, academic departments have become poor arbiters. They are loathe to say no to a student's progression, in part because the student is a low cost laborer for an investigator in the department. Until the departments bear some of the cost of this training, they will not be good stewards of federal training dollars.
The NIH would be right to try to clamp down on Ph.D. training funding at the federal level (among other poorly optimized investments). In the global perspective a Ph.D. pipeline of a bore so vastly wider than the R01 funding pipeline is a low yield investment to society at large. I say this as someone who, as a future investigator who came through in the time of high funding of trainees, will be hurt by the loss of cheap labor welling up below me. However this affects much more than any future lab bottom line, it also influences the credibility of the scientific enterprise as a whole. It is an incredible privilege to do what we do, and to keep doing it we must be good stewards of the public's trust.
As a postscript, should I mention that I don't really believe in the Ph.D. as a degree anyway? Degrees end up being carrots or sticks that PIs can wield against their trainees when they have already become excellent candidates on the job market. Viable Ph.D.s will already have publications and poster presentations to prove the merits of their productivity. The Ph.D. defense has become a formality that we should abolish. Degrees in general, and the Ph.D. in particular, too often become a tool to preserve privilege of elites, and become a dangerous replacement for real competence.
Many opinions were aired, but there were a few insights that I thought resounded most clearly above the hubbub of what is, of course, a very political question.
1) Cull the herd: Do not allow students to continue Ph.D. degrees if they are unlikely to be successful. There was one individual who offered a clear comparison between what is done in the biomedical sciences and what is done in economics. The benefit of this are, at the very least, less delusion from students thinking that they might be able to go on in academia and less crush for grants when Ph.D.'s go on to apply for faculty positions. The cost is less cheap labor available to PIs in the biomedical sciences.
2) Numerator vs. Denominator: How do we avoid the simple problem of increasing Ph.D.s and constant (or decreasing!) funding. Are there operational efficiencies that can be achieved or some kind of restructuring of granting mechanisms that allow for continued scientific output, career stability, and stable costs? Interestingly the fundamental nature of this question was often overtaken by discussions of how nice individual programs might be in terms of some elements of this equation (e.g. a career track for core lab administrators might permit both continued scientific output and career stability... but at what cost?)
What is the cost, what is the return? In the end, this is an economic question. We in science are at risk of confusing the return to ourselves as the return to the investor. As academics, perhaps this is forgivable. We are, after all, obsessed with the deep questions of science. How does the cell work? How does the genome lock in a frozen, endlessly replicated organic molecule, the blueprint of life? How does cancer evolve? How can we better treat disease? What is good for us in our quest must be good for society, should we successfully answer these questions.
But the fact of the matter is much more complex. Graduate school is not a zero sum game, nor is economics. But the allocation of federal funding, sadly, is. The questions above are never fully answered, they are only answered in part. An attention to yield is paramount. As should be obvious, the yield equation is not the sum of impact factors of all the papers written by a set of investigators. It is the sum of the benefits to the individuals of the nation (the world?) over the centuries.
Whether or not we have stable careers is irrelevant latter sum of benefits, except inasmuch as stable careers allow researchers to do fundamental work (I think that they do, but to less extent than we might claim). What is more relevant, as far as federal funding is concerned, is the fact that a Ph.D. that is underutilized may be a wasted investment. In addition to the costs of funding the Ph.D., society has lost the other work that that presumably intelligent and capable individual could have achieved in that time in another line of work. Depending on the person, this may represent hundreds of thousands of dollars of loss. If they by exiting the academic track in fact exit the job market completely, the loss may even reach the millions.
The fact is, academic departments have become poor arbiters. They are loathe to say no to a student's progression, in part because the student is a low cost laborer for an investigator in the department. Until the departments bear some of the cost of this training, they will not be good stewards of federal training dollars.
The NIH would be right to try to clamp down on Ph.D. training funding at the federal level (among other poorly optimized investments). In the global perspective a Ph.D. pipeline of a bore so vastly wider than the R01 funding pipeline is a low yield investment to society at large. I say this as someone who, as a future investigator who came through in the time of high funding of trainees, will be hurt by the loss of cheap labor welling up below me. However this affects much more than any future lab bottom line, it also influences the credibility of the scientific enterprise as a whole. It is an incredible privilege to do what we do, and to keep doing it we must be good stewards of the public's trust.
As a postscript, should I mention that I don't really believe in the Ph.D. as a degree anyway? Degrees end up being carrots or sticks that PIs can wield against their trainees when they have already become excellent candidates on the job market. Viable Ph.D.s will already have publications and poster presentations to prove the merits of their productivity. The Ph.D. defense has become a formality that we should abolish. Degrees in general, and the Ph.D. in particular, too often become a tool to preserve privilege of elites, and become a dangerous replacement for real competence.
Wednesday, June 11, 2014
A New Conversation
The coauthor of this blog and I recently published and op-ed in the News and Observer about the state of modern cancer research and our thoughts about how young investigators can best join in to move research forward. We focus on the applied side of the process. We ask the question: what can we do to develop chemotherapy we can believe in? The process of writing the article got us excited about participating in the public conversation about medicine, research, and cancer.
To that end, I will be publishing more to this blog. We will include more in depth discussion here of articles that we we write for other media. We're currently working on more Op-Ed articles, but also other article formats that might find a home here on the blog. This may or may not have any lasting efficacy, but all good ideas deserve a proper experiment!
To that end, I will be publishing more to this blog. We will include more in depth discussion here of articles that we we write for other media. We're currently working on more Op-Ed articles, but also other article formats that might find a home here on the blog. This may or may not have any lasting efficacy, but all good ideas deserve a proper experiment!
Friday, January 20, 2012
So researchGate is kind of an interesting idea. As is this this New York Times article on the future of science publishing. I happen to agree that science publishing is due for a major makeover, but I don't think its replacement is out there yet. PLoS One has a lot going for it, as do other open access journals, but they don't bring us closer to the thing I really want to see: real time science publication.
I'm not totally sure that it's a desirable end, since a lot of what gets done in paper writing (careful description of methods, bringing together disparate data into one story, filling in logical holes with supporting experiments) only gets done because of the hurdle of publication. But imagine how many small, interesting, positive results never get published because they don't fit in with such a complete narrative required by a paper. Or, for that matter, the number of negative results.
What if, at the end of the day, everyone just blogged their science. You'd put a blurb about your methods, a blurb about your results, maybe a picture of your gel, sequencing results, graph, whatever, and let it run free on the internet. I can think of 2 or 3 figures that I plan to put into a forthcoming paper that I wish I didn't have to wait to share.
What are the down sides? Well a blog is less permanent than a publication, which gets archived and saved in about a hundred different places. It also is hard to put on your CV. Moreover, do people really want to go to someone's blog to learn what their lab is doing, or would they much prefer a scientific paper, which is sort of a digest version? Finally, does it actually advance science if everyone can know what everyone else is doing at all times? Would people continue to have the drive to publish first to avoid being scooped? Would there be even less reward for replicating past results, since you could in theory check to make sure you were the only one working in your area?
The expanding ecosystem is not without problems, but it's interesting to see what is created in the years ahead.
I'm not totally sure that it's a desirable end, since a lot of what gets done in paper writing (careful description of methods, bringing together disparate data into one story, filling in logical holes with supporting experiments) only gets done because of the hurdle of publication. But imagine how many small, interesting, positive results never get published because they don't fit in with such a complete narrative required by a paper. Or, for that matter, the number of negative results.
What if, at the end of the day, everyone just blogged their science. You'd put a blurb about your methods, a blurb about your results, maybe a picture of your gel, sequencing results, graph, whatever, and let it run free on the internet. I can think of 2 or 3 figures that I plan to put into a forthcoming paper that I wish I didn't have to wait to share.
What are the down sides? Well a blog is less permanent than a publication, which gets archived and saved in about a hundred different places. It also is hard to put on your CV. Moreover, do people really want to go to someone's blog to learn what their lab is doing, or would they much prefer a scientific paper, which is sort of a digest version? Finally, does it actually advance science if everyone can know what everyone else is doing at all times? Would people continue to have the drive to publish first to avoid being scooped? Would there be even less reward for replicating past results, since you could in theory check to make sure you were the only one working in your area?
The expanding ecosystem is not without problems, but it's interesting to see what is created in the years ahead.
Sunday, January 8, 2012
Well intentioned mediocrity?
Opposing the bill that started the National Cancer Institute, James Watson, the loose cannon co-discoverer of DNAs structure said:
… we must reject the notion that we will be lucky. … Instead we will be witnessing a massive expansion of well-intentioned mediocrity.
(credit: The Emperor of all Maladies)
Can we say, truthfully, that he was wrong? Or has the cure for cancer been a long road that will soon bear its full fruit?
Friday, November 18, 2011
Flip the Conference
People who are a fan of the khan academy might know about the concept of "flipping the classroom". The simple idea of this technique is having students watch lecture videos as homework and do problems in the classroom, rather than the reverse.
I wonder whether this same method could be applied to conferences, which often result in long, dull talks followed by interesting questions and discussions. One could imagine recorded talks made available to conference attendees a few days in advance followed by a conference entirely devoted to activities designed to promote interaction between researchers, such as poster sessions, discussion panels, small workshops and even debates. Why spend a day watching a big screen in front of the lecture hall in some convention center when we can just as easily do that in our bathrobe at home?
I wonder whether this same method could be applied to conferences, which often result in long, dull talks followed by interesting questions and discussions. One could imagine recorded talks made available to conference attendees a few days in advance followed by a conference entirely devoted to activities designed to promote interaction between researchers, such as poster sessions, discussion panels, small workshops and even debates. Why spend a day watching a big screen in front of the lecture hall in some convention center when we can just as easily do that in our bathrobe at home?
Thursday, November 17, 2011
TCGA Datasplosion
I'm at the TCGA meeting in Washington DC, and am being blown away by the enormous amount of data and complicated analysis of that data. It's phenomenal how much sequencing, morphometric and related data have been accrued in a wide variety of tumor types. The analyses are pretty interesting, but it's also surprising how much previously known biology comes out again and again. I can't tell if people are reporting results that we already know because it's safer, or because there's not much new in the data. Either way, it's clear that for these results to be useful we will need much more clinically relevant annotation and many more tumors. In a future where all patients have sequenced tumors, perhaps we will see much more clinically applicable conclusions.
Friday, October 28, 2011
Micro-Science-Blogging
Here's a crazy idea, what if scientists posted results of their studies for the year, month, week, or hell, even their exciting result of the day, to YouTube? What if we blogged our science?
Let's say, for sake of argument, that I have an interesting result, not yet ready for publication in a journal but certainly worthy of the attention and maybe comments from other researchers. On the one hand, I could work on that idea further until it's ready for a "real" publication. That would probably involve some missteps, maybe some time lost (open the door for scooping), but will also make for a more complete product when it is actually released.
On the other hand, I could open myself up for comments at a very early stage. Not only would the online "timestamp" validate my status as first to the result immediately, it would also give others a chance to comment, criticize, and potentially even initiate their own projects. When the project reaches a later stage I will have already incorporated "reviewer" comments into my manuscript, which will remain just as valuable as a formal report of the work.
Perhaps such a system could work on a by-lab basis. Labs could become, in a weird way, their own publishing houses. Weekly updates on results could feed into the RSS feed of other labs. Just a thought
Saturday, October 22, 2011
If anyone is wondering what regulation capture by industry interests looks like, this is it:
A Life in Energy and (Therefore) Politics (wsj)
The energy industry is mired with so many competing regulatory and industry interests that calling it an "energy market" is a complete misnomer. Note the intriguing comments made later on in the article about some "dirty: coal plants being used as bargaining chips by companies against state and local politicians who, having granted monopolies, are now beholden to those companies.
It's a real fine mess.
A Life in Energy and (Therefore) Politics (wsj)
The energy industry is mired with so many competing regulatory and industry interests that calling it an "energy market" is a complete misnomer. Note the intriguing comments made later on in the article about some "dirty: coal plants being used as bargaining chips by companies against state and local politicians who, having granted monopolies, are now beholden to those companies.
It's a real fine mess.
Wednesday, October 12, 2011
The Rational Optimist and Ricardo
I'm about halfway through The Rational Optimist and have to say that it's a bolstering case for the phenomenal power of markets and trade in human society. It's not exactly erudite, but it's a well versed look at the way trade and specialization have massively improved the lot of the average human in the past few million years and enabled us to grow to a population of several billion. Some of the points I like the most revolve around Ricardo's Law of Comparative Advantage, which the book describes as "the only result of social science that is both surprising and true". In honor of the recent free trade agreement between the US and several countries, I want to take a moment on this little law.
Briefly, the law states that if two individuals, nations, companies, whatever, have a different relative ability to produce a pair of different items, then one or both of the actors will be better off if they trade. So, for example, even if France is better at making both wine and cheese than England (probably true), both England and France will still benefit from trade if the relative cost of producing wine vs. cheese in the two countries are different.
What if France makes wine much more efficiently than cheese, and England does as well? It turns out, it doesn't matter. If the tradeoffs of wine vs. cheese are just different, that's all it takes for both countries to benefit.
What's marvelous about The Rational Optimist is that it couches these modern conceptions of economics in the ancient environment, before they were even realized. Let's say I make spears incredibly well, and I also run down and spear bison incredibly well (probably not true). I take a day to make a high quality spear, and a day to run down a bison. You, sadly, are terrible at both, taking 2 days to make a high quality spear, and 4 days to run down a bison.
Now we could both spend time in our own little spheres, making spears and running around all day. At the end of 6 days, you would have one spear, which you will have used bringing down a bison. I will have made 3 spears and brought down 3 bison.
After those 6 days, you have a brilliant idea. A brilliant idea that will keep you from running around all day and instead put my hands at work for you. Let's say that the next 2 days look a lot like the 6 days before. I make a spear, and go skewer a bison with it. You're just be polishing off that first spear when I bring the bison home.
But now you throw your incredibly brilliant plan into effect. "Ugg!" you say, (we're cave man, that's the my moniker) "I'll give you that spear for just 1/3 of your next bison!".
I think it through. 1/3 of my next bison? But I can have all the bison if I make the spear myself. But if I make my next hunting spear myself, I have to sit around all day making it, and won't be able to hunt until tomorrow. If I take this spear, I can hunt now. Let's see... 1 bison in 2 days, or 2/3rds of a bison in 1 day. I might not be able to figure out the fractions, but I know that I'm getting more. I take your spear, head out for the hunt, and that night I come back with my prize. You get 1/3 of a Bison, and never had to run around on the hot plains.
I won't belabor the point, but if you work out the math, over 6 days with the occasional spear trading hands, you still get your one bison by making 3 spears instead of 1 spear and one hunting trip. I am doing a lot more running, but I get 3 hunting trips without having to stop to make spears, and can make one hunting trip with my own spear and still have a day to spare. Calculated out, I get the same 3 bison, and a free vacation day!
Consider that this example is the most meager form of Ricardo's comparative advantage. Imagine that after a year of this arrangement, you get absurdly good at spear making. You've been doing it every day of the week after all, and you've had a chance to try some things out. Now, after figuring out how to use a rock to sharpen the tips and finding the best source of branches in the woods, you're turning out 2 or 3 a day! You're still only trading one for 1/3 of a buffalo, but now you can stock up on spears the first 2 days of the week and spend the rest of the time figuring out how to corral a buffalo calf and save it for later.
Clearly I've just sketched the barest bones of the argument here, and I highly recommend the book for a better treatment. The point, though, is that specialization and trade are massively powerful ways to improve our own lives and those of our trading partners. Imagine, in your own life, how difficult it would be to independently create all the wonderful artifacts of human productivity you enjoy. What you get out of voluntary trade makes possible the life you lead today.
Briefly, the law states that if two individuals, nations, companies, whatever, have a different relative ability to produce a pair of different items, then one or both of the actors will be better off if they trade. So, for example, even if France is better at making both wine and cheese than England (probably true), both England and France will still benefit from trade if the relative cost of producing wine vs. cheese in the two countries are different.
What if France makes wine much more efficiently than cheese, and England does as well? It turns out, it doesn't matter. If the tradeoffs of wine vs. cheese are just different, that's all it takes for both countries to benefit.
What's marvelous about The Rational Optimist is that it couches these modern conceptions of economics in the ancient environment, before they were even realized. Let's say I make spears incredibly well, and I also run down and spear bison incredibly well (probably not true). I take a day to make a high quality spear, and a day to run down a bison. You, sadly, are terrible at both, taking 2 days to make a high quality spear, and 4 days to run down a bison.
Now we could both spend time in our own little spheres, making spears and running around all day. At the end of 6 days, you would have one spear, which you will have used bringing down a bison. I will have made 3 spears and brought down 3 bison.
After those 6 days, you have a brilliant idea. A brilliant idea that will keep you from running around all day and instead put my hands at work for you. Let's say that the next 2 days look a lot like the 6 days before. I make a spear, and go skewer a bison with it. You're just be polishing off that first spear when I bring the bison home.
But now you throw your incredibly brilliant plan into effect. "Ugg!" you say, (we're cave man, that's the my moniker) "I'll give you that spear for just 1/3 of your next bison!".
I think it through. 1/3 of my next bison? But I can have all the bison if I make the spear myself. But if I make my next hunting spear myself, I have to sit around all day making it, and won't be able to hunt until tomorrow. If I take this spear, I can hunt now. Let's see... 1 bison in 2 days, or 2/3rds of a bison in 1 day. I might not be able to figure out the fractions, but I know that I'm getting more. I take your spear, head out for the hunt, and that night I come back with my prize. You get 1/3 of a Bison, and never had to run around on the hot plains.
I won't belabor the point, but if you work out the math, over 6 days with the occasional spear trading hands, you still get your one bison by making 3 spears instead of 1 spear and one hunting trip. I am doing a lot more running, but I get 3 hunting trips without having to stop to make spears, and can make one hunting trip with my own spear and still have a day to spare. Calculated out, I get the same 3 bison, and a free vacation day!
Consider that this example is the most meager form of Ricardo's comparative advantage. Imagine that after a year of this arrangement, you get absurdly good at spear making. You've been doing it every day of the week after all, and you've had a chance to try some things out. Now, after figuring out how to use a rock to sharpen the tips and finding the best source of branches in the woods, you're turning out 2 or 3 a day! You're still only trading one for 1/3 of a buffalo, but now you can stock up on spears the first 2 days of the week and spend the rest of the time figuring out how to corral a buffalo calf and save it for later.
Clearly I've just sketched the barest bones of the argument here, and I highly recommend the book for a better treatment. The point, though, is that specialization and trade are massively powerful ways to improve our own lives and those of our trading partners. Imagine, in your own life, how difficult it would be to independently create all the wonderful artifacts of human productivity you enjoy. What you get out of voluntary trade makes possible the life you lead today.
Monday, October 10, 2011
Minimum Publishable Unit
What the hell, I'll post again this year.
When is the right time to publish? Is it when you have an interesting result? Is it when your result is "complete", zipped up, lock-tight, slam-dunk? Is it when the result seems more likely true than not? Is it when you want to recruit others to the cause? Is it when you think it's good enough to impress people, and impress them in the amount that you need them to be impressed for your career to advance?
Sadly, I think the answer is often that last on my list. Publications are the the currency of science, and the only hard paper that demonstrates productivity on the public stage. Science is a big establishment, and our individual spheres are often tightly circumscribed, though they may seem vast from the inside. If I am applying for a job, and I have a Nature paper or a Cell paper (not happening any time soon, but let's just imagine for a moment), then I can put that on the desk and point to it. Say my competitor has a paper in PLoS Genetics. Still good, still the same paper, but different brand. I'd carry more weight with the Nature paper.
That brand seeking behavior is nothing new, and it's not actually the problem in itself. The problem is when the brand takes on a life of its own. If everyone buys Nike shoes because they are flashy and Nike, and not because the Nike brand speaks to quality, then where can you expect the quality to go? Similarly if Nature papers are sought because they are flashy, and hard to get, where does Nature go from there? Will Nature continue to speak to the quality of the underlying science?
My fundamental problem with the big name paper is that the results are often big and sweeping. They are rarely circumscribed, but elegant and well thought out. Imagine a demonstration that a single class of proteins perform some specific catalytic or signaling function. This in itself is worthy of being shared with the scientific community, as an informative work. But to get the Nature paper you need to show the protein is relevant in disease X, and that if you inhibit the catalytic activity you cure warts, cancer, and heart disease.
Of course inhibition of the protein won't turn out to actually cure warts, cancer and heart disease. Somewhere along the line a wrinkle will have been ignored. Some control, that in hindsight will be obvious, won't have been done. This is inevitable in the sort of expansive manuscripts that top journals demand.
I argue we should be more focused on rapid dissemination of our research, and broad feedback from the community early on in validating a result. I might run a smart, genome-wide screen, but you might have a better idea of how to interpret it. Someone else might recognize an important statistical error. Most importantly, someone else can *replicate* the experimental results early, before I truck on down the road with faulty assumptions.
The problem is not that the "minimum publishable unit" is too small, it's that it's too big, and that we focus on the papers and the journals and not on the results themselves. In the end, a Nature paper that's wrong is worth less than a small time journal article that's right. The Nature paper that's wrong may actually have negative value, having led labs down the wrong road. Next time you hear someone say that they have a Cell paper under their belt, ask next whether the result was replicated. Ask whether the result has, in fact, been important. Remember to value not the scientific articles, but the science itself.
When is the right time to publish? Is it when you have an interesting result? Is it when your result is "complete", zipped up, lock-tight, slam-dunk? Is it when the result seems more likely true than not? Is it when you want to recruit others to the cause? Is it when you think it's good enough to impress people, and impress them in the amount that you need them to be impressed for your career to advance?
Sadly, I think the answer is often that last on my list. Publications are the the currency of science, and the only hard paper that demonstrates productivity on the public stage. Science is a big establishment, and our individual spheres are often tightly circumscribed, though they may seem vast from the inside. If I am applying for a job, and I have a Nature paper or a Cell paper (not happening any time soon, but let's just imagine for a moment), then I can put that on the desk and point to it. Say my competitor has a paper in PLoS Genetics. Still good, still the same paper, but different brand. I'd carry more weight with the Nature paper.
That brand seeking behavior is nothing new, and it's not actually the problem in itself. The problem is when the brand takes on a life of its own. If everyone buys Nike shoes because they are flashy and Nike, and not because the Nike brand speaks to quality, then where can you expect the quality to go? Similarly if Nature papers are sought because they are flashy, and hard to get, where does Nature go from there? Will Nature continue to speak to the quality of the underlying science?
My fundamental problem with the big name paper is that the results are often big and sweeping. They are rarely circumscribed, but elegant and well thought out. Imagine a demonstration that a single class of proteins perform some specific catalytic or signaling function. This in itself is worthy of being shared with the scientific community, as an informative work. But to get the Nature paper you need to show the protein is relevant in disease X, and that if you inhibit the catalytic activity you cure warts, cancer, and heart disease.
Of course inhibition of the protein won't turn out to actually cure warts, cancer and heart disease. Somewhere along the line a wrinkle will have been ignored. Some control, that in hindsight will be obvious, won't have been done. This is inevitable in the sort of expansive manuscripts that top journals demand.
I argue we should be more focused on rapid dissemination of our research, and broad feedback from the community early on in validating a result. I might run a smart, genome-wide screen, but you might have a better idea of how to interpret it. Someone else might recognize an important statistical error. Most importantly, someone else can *replicate* the experimental results early, before I truck on down the road with faulty assumptions.
The problem is not that the "minimum publishable unit" is too small, it's that it's too big, and that we focus on the papers and the journals and not on the results themselves. In the end, a Nature paper that's wrong is worth less than a small time journal article that's right. The Nature paper that's wrong may actually have negative value, having led labs down the wrong road. Next time you hear someone say that they have a Cell paper under their belt, ask next whether the result was replicated. Ask whether the result has, in fact, been important. Remember to value not the scientific articles, but the science itself.
Saturday, October 8, 2011
Irritated in Chapel Hill
The scientific establishment is Democratic.
News to anyone? Probably not. After all, Democrats by and large promote increases in NIH and NSF funding, tout the importance of the scientific consensus, and so on. But when one is, oh, say, not a Democrat, and trying to do science within the field as it currently exists, there are bound to be frustrating moments.
Take, for example, two seminars in a year with national Democratic representatives. Let me add that these seminars are ambiguously mandatory, with a sign in sheet at many, though not all events, and that they are catered by the program. Imagine a room full of decidedly left leaning individuals with their representative on stage. These reps, so far as I can tell, have nothing to talk about but their political leanings and aspirations. And they'll have all the time in the world to tell you how important they think it is to support, Education and (National) Health Care. They'll also mention how they plan, at the Federal level, to fund Science, Education and (National) Health Care. Nevermind where the money is coming from...
Now I'm not saying no seminar has had a Independent, Republican, or otherwise non-democratically leaning individual who has weighed in, where asked, on political issues. It's just that I have yet to see such an individual given an open forum with a mandate to talk about politics. And perhaps I am being unfair, since I can't imagine any such speakers wanting to wade into the issues of a limited Federal role in that crowd. The Feds are our lifeblood, as it currently stands, and I am one of the few who bites the hand that feeds him. Hell, I have even applied for Federal grants, and benefit from State money given to our Cancer Center, so who am I to complain?
But of course that is exactly the ratchet of Federal authority, ever gaining constituents through patronage, that erodes limited government. I have to live within the system as it exists, even as I argue for its change, as I do here. I just wish that this group of ostensibly deeply thinking individuals were less overwhelmingly game to sign on with whoever holds the purse strings. And I wish they didn't strongly (though perhaps unintentionally) broadcast their inclinations from the top down.
I hate being a naysayer, and I hate biting back at a program that has given me a great deal. But in case people are suffering under the illusion that the MD/PhD program at UNC has built a scientific consensus that you should vote for David Price and Kay Hagen , I just want to make it clear that it has not.
News to anyone? Probably not. After all, Democrats by and large promote increases in NIH and NSF funding, tout the importance of the scientific consensus, and so on. But when one is, oh, say, not a Democrat, and trying to do science within the field as it currently exists, there are bound to be frustrating moments.
Take, for example, two seminars in a year with national Democratic representatives. Let me add that these seminars are ambiguously mandatory, with a sign in sheet at many, though not all events, and that they are catered by the program. Imagine a room full of decidedly left leaning individuals with their representative on stage. These reps, so far as I can tell, have nothing to talk about but their political leanings and aspirations. And they'll have all the time in the world to tell you how important they think it is to support, Education and (National) Health Care. They'll also mention how they plan, at the Federal level, to fund Science, Education and (National) Health Care. Nevermind where the money is coming from...
Now I'm not saying no seminar has had a Independent, Republican, or otherwise non-democratically leaning individual who has weighed in, where asked, on political issues. It's just that I have yet to see such an individual given an open forum with a mandate to talk about politics. And perhaps I am being unfair, since I can't imagine any such speakers wanting to wade into the issues of a limited Federal role in that crowd. The Feds are our lifeblood, as it currently stands, and I am one of the few who bites the hand that feeds him. Hell, I have even applied for Federal grants, and benefit from State money given to our Cancer Center, so who am I to complain?
But of course that is exactly the ratchet of Federal authority, ever gaining constituents through patronage, that erodes limited government. I have to live within the system as it exists, even as I argue for its change, as I do here. I just wish that this group of ostensibly deeply thinking individuals were less overwhelmingly game to sign on with whoever holds the purse strings. And I wish they didn't strongly (though perhaps unintentionally) broadcast their inclinations from the top down.
I hate being a naysayer, and I hate biting back at a program that has given me a great deal. But in case people are suffering under the illusion that the MD/PhD program at UNC has built a scientific consensus that you should vote for David Price and Kay Hagen , I just want to make it clear that it has not.
Sunday, October 31, 2010
The Hazard of High Throughput
We live in a high throughput age. Science is no exception. Microarrays, high throughput sequencing and spectrophotometers generate data on a scale and scope that would have taken years, decades, or centuries with the the old generation of technology. We are generating data on a scale that could not be conceived of a decade ago.
Great power... let's see what comes next....
The challenge, of course, is with the analysis. The data now generated is so massive that it cannot be simultaneously visualized or processed in its raw form. Nor can t-tests get you where you want to go with statistical analysis, so many are the comparisons being made.
Enter the world of the Bonferoni correction and the Benjamini-Hochberg false discovery rate. These statistical methods allow us to sift through such enormous data sets to focus on results that are significantly different from random expectation.
The hazard comes with these methods' complexity and somewhat obscure statistical assumptions. Many scientists are very well versed in the hypotheses of their discipline, but less so on the mathematics. There are so many ways to go wrong in applying these methods in a cookie cutter way that it boggles the mind. Along the lines of "Correlation implies causation" there are other such gems as: "Difference in significance does not imply significant difference".
This last mistake was featured prominently in stage 1 of a statistical analysis in a manuscript from a good lab that just passed my boss' desk. This lab has produced prominent publications in the past, and I was surprised to see this in their analysis.
What most surprised me was that the statistical method, buried deep within the methods section at the end of the manuscript, did not arouse the ire of my boss or the other lab member who read the paper. It was viewed as a good enough an answer to a tough problem. That, plus the prominence of the last author, led to a minor note somewhere in the review.
Reviews don't have dissenting opinions, but let me put one here. Statistical methods are important. So important that, in a paper that uses a high throughput method at its start, they often form the backbone for every follow up experiment. They should not be relegated to a footnote in the back and they should, wherever possible, be declared before the data are even generated, to avoid the nefarious problems of overfitting.
Papers that use statistic need statistically minded reviewers. If we aren't careful, we'll be fooled by randomness.
Great power... let's see what comes next....
The challenge, of course, is with the analysis. The data now generated is so massive that it cannot be simultaneously visualized or processed in its raw form. Nor can t-tests get you where you want to go with statistical analysis, so many are the comparisons being made.
Enter the world of the Bonferoni correction and the Benjamini-Hochberg false discovery rate. These statistical methods allow us to sift through such enormous data sets to focus on results that are significantly different from random expectation.
The hazard comes with these methods' complexity and somewhat obscure statistical assumptions. Many scientists are very well versed in the hypotheses of their discipline, but less so on the mathematics. There are so many ways to go wrong in applying these methods in a cookie cutter way that it boggles the mind. Along the lines of "Correlation implies causation" there are other such gems as: "Difference in significance does not imply significant difference".
This last mistake was featured prominently in stage 1 of a statistical analysis in a manuscript from a good lab that just passed my boss' desk. This lab has produced prominent publications in the past, and I was surprised to see this in their analysis.
What most surprised me was that the statistical method, buried deep within the methods section at the end of the manuscript, did not arouse the ire of my boss or the other lab member who read the paper. It was viewed as a good enough an answer to a tough problem. That, plus the prominence of the last author, led to a minor note somewhere in the review.
Reviews don't have dissenting opinions, but let me put one here. Statistical methods are important. So important that, in a paper that uses a high throughput method at its start, they often form the backbone for every follow up experiment. They should not be relegated to a footnote in the back and they should, wherever possible, be declared before the data are even generated, to avoid the nefarious problems of overfitting.
Papers that use statistic need statistically minded reviewers. If we aren't careful, we'll be fooled by randomness.
Sunday, October 17, 2010
Formulary of Ten
Here's a quick question: if you had to trim the formulary of a hospital down to just 10 adult meds, what would they be? I'm sure a careful and informed individual could make a comprehensive choice based on "Quality of Life Year" information and some epidemiology of the area. Here's what I am thinking:
1) Morphine
2) Aspirin
3) One or two antibiotics (Guess who doesn't remember micro enough to know which ones? I am guessing Cipro/levo and Vancomycin)
5) Insulin
6) Beta-blocker
7) A statin
8) Doxorubicin (What my boss called: the one chemo you'd bring to a desert island)
9) Warfarin
10) Lidocain
Thoughts? It might be a little over-covered for heart disease and under-covered for diabetes. Also, is it even worth it to have a chemo agent, from a utilitarian perspective? I know that's heretical from someone who is interested in cancer research, but how much life are you really adding with one therapy alone?
Also: does Morphine/lidocain deserve a place? Pain management is important, and I didn't want to leave it out. But when there are other diseases being left uncovered, should this be a top priority?
Anyone care to offer an opinion? It's an interesting way of thinking about priorities in U.S. medicine, and pondering how much we take for granted. What medicines are those that we can't make due without?
1) Morphine
2) Aspirin
3) One or two antibiotics (Guess who doesn't remember micro enough to know which ones? I am guessing Cipro/levo and Vancomycin)
5) Insulin
6) Beta-blocker
7) A statin
8) Doxorubicin (What my boss called: the one chemo you'd bring to a desert island)
9) Warfarin
10) Lidocain
Thoughts? It might be a little over-covered for heart disease and under-covered for diabetes. Also, is it even worth it to have a chemo agent, from a utilitarian perspective? I know that's heretical from someone who is interested in cancer research, but how much life are you really adding with one therapy alone?
Also: does Morphine/lidocain deserve a place? Pain management is important, and I didn't want to leave it out. But when there are other diseases being left uncovered, should this be a top priority?
Anyone care to offer an opinion? It's an interesting way of thinking about priorities in U.S. medicine, and pondering how much we take for granted. What medicines are those that we can't make due without?
Thursday, October 14, 2010
Peer Review
If you've spent some time in graduate school you might have learned a thing or two about the open secret about peer review: it's not entirely done by peers and it's not always that thorough of a review.
Like it or not, grad students often find these reviews on their desk. I've heard of PIs leaving the job to grad students alone, though this does not occur in my lab. It's understandable, though, that this might happen. PIs have huge demands on their time. As invaluable as their opinions are, there is no way some PIs can field every manuscript. Apparently they have their own concerns about peer review (some of which apparently involves colorful dinos)
But what of the reviews performed by graduate students? There are two ways of looking at the duty of peer review.
Looked at one way, a peer review is a learning opportunity. Often the material details development on the edge of some field that contacts at least tangentially upon the student's research. It's a chance to see another lab's raw, early draft manuscript, and learn what merits publication in high level journals and what does not. It is a learning experience, and a chance to contribute to the body of science as a whole.
But let's just step out of the shiny world of gumdrops and candy canes for a moment. Peer review can be an inane chore. While students provide some value added to the journal and the author of the manuscript, it's harder to see where the review process benefits their progression in the ladder. Their role in the review is, effectively, anonymous, and comes with no honor, distinctions, gold stars, pats on the head, brownie points or first author publications.
Stated simply, there is no clear match between incentives and the quality of the peer review. Mistakes are often buried somewhere deep in the unending, jargon-filled paragraphs of the methods section. Whether a grad student takes the effort to check these methods, line by line, comes down to a question of how many hours (if any) of sleep they might prefer to have that evening.
Where's the incentive to dig in? Some grad students seem to possess a deep personal drive to throw other scientists under the bus, but it's probably a minority at most institutions. We can't rely on pure sadism to drive the scientific engine. There must be a way to reward careful and well considered reviews, particularly where they find obscure errors and tenuous methods.
I don't claim to know what the reward should be or how it could be structured. I've considered the possibility that confidential peer review is a mistake, and that publications should instead be edited by the journal seeking to publish. If Nature wants the value added of an expert opinion in the field, let them pay for it! Certainly they demand payment for their subscriptions, so why should their product be provided for free?
At risk of shifting to a seemingly radical alternative, perhaps open access and open comment system is the way to go. Take all comers that pass a basic editorial spot check, and allow insightful, observational comments to come from the community. Those comments can then be tied to the reputation of those who make them. Great insights can be noticed, and unnecessary bickering (I'm looking at you, reviewer #3), can be ignored. Online systems for grading and sorting comments based on reputation systems exist in many forms. Perhaps its time to turn them loose on science.
Concurrently, give labs the hard task of determining their own publication threshold. Perhaps more self review will go on in-house if authors know that they can publish whatever they please, and that they'd better get it right the first time. Perhaps some systems could allow papers to have a 'versioning' system, wherein they could be updated (to a point) to reflect public comment.
I don't claim that pay-to-review or open-comment systems solve the problems inherent in the current publication regime, but I think they deserve consideration. Let us at least recognize that good work deserves good incentives, and that the adding motivation to peer-review can only improve our scientific rigor.
Like it or not, grad students often find these reviews on their desk. I've heard of PIs leaving the job to grad students alone, though this does not occur in my lab. It's understandable, though, that this might happen. PIs have huge demands on their time. As invaluable as their opinions are, there is no way some PIs can field every manuscript. Apparently they have their own concerns about peer review (some of which apparently involves colorful dinos)
But what of the reviews performed by graduate students? There are two ways of looking at the duty of peer review.
Looked at one way, a peer review is a learning opportunity. Often the material details development on the edge of some field that contacts at least tangentially upon the student's research. It's a chance to see another lab's raw, early draft manuscript, and learn what merits publication in high level journals and what does not. It is a learning experience, and a chance to contribute to the body of science as a whole.
But let's just step out of the shiny world of gumdrops and candy canes for a moment. Peer review can be an inane chore. While students provide some value added to the journal and the author of the manuscript, it's harder to see where the review process benefits their progression in the ladder. Their role in the review is, effectively, anonymous, and comes with no honor, distinctions, gold stars, pats on the head, brownie points or first author publications.
Stated simply, there is no clear match between incentives and the quality of the peer review. Mistakes are often buried somewhere deep in the unending, jargon-filled paragraphs of the methods section. Whether a grad student takes the effort to check these methods, line by line, comes down to a question of how many hours (if any) of sleep they might prefer to have that evening.
Where's the incentive to dig in? Some grad students seem to possess a deep personal drive to throw other scientists under the bus, but it's probably a minority at most institutions. We can't rely on pure sadism to drive the scientific engine. There must be a way to reward careful and well considered reviews, particularly where they find obscure errors and tenuous methods.
I don't claim to know what the reward should be or how it could be structured. I've considered the possibility that confidential peer review is a mistake, and that publications should instead be edited by the journal seeking to publish. If Nature wants the value added of an expert opinion in the field, let them pay for it! Certainly they demand payment for their subscriptions, so why should their product be provided for free?
At risk of shifting to a seemingly radical alternative, perhaps open access and open comment system is the way to go. Take all comers that pass a basic editorial spot check, and allow insightful, observational comments to come from the community. Those comments can then be tied to the reputation of those who make them. Great insights can be noticed, and unnecessary bickering (I'm looking at you, reviewer #3), can be ignored. Online systems for grading and sorting comments based on reputation systems exist in many forms. Perhaps its time to turn them loose on science.
Concurrently, give labs the hard task of determining their own publication threshold. Perhaps more self review will go on in-house if authors know that they can publish whatever they please, and that they'd better get it right the first time. Perhaps some systems could allow papers to have a 'versioning' system, wherein they could be updated (to a point) to reflect public comment.
I don't claim that pay-to-review or open-comment systems solve the problems inherent in the current publication regime, but I think they deserve consideration. Let us at least recognize that good work deserves good incentives, and that the adding motivation to peer-review can only improve our scientific rigor.
Grad Year 1, Tiny Learnings
Tiny learnings from grad year 1, stated briefly, for my own purposes. Trite? Maybe. Suck it up.
Mentorship is everywhere, always take the opportunity to meet people doing different things in different labs. The guy down the hall may solve the problem that's been killing you all month. That actually happened to me today, and it was great.
Look out for #1, a little bit. Cognizant of the goal being science, remember that not everyone has your interests at heart. Example: you are very cheap labor for your PI and he/she isn't itching to see you leave. This cuts both ways, remember that you might not be respecting someone else's need to look out for #1. Example: Your PI has duties besides being on-call for you 24/7.
Related Fact: When your PI says it's easy, it's probably hard. If he says it's doable, it'll probably consume all of your attention for as long as you want to work on it. Remember that you are missing years of experience on the person you're talking to.
Write up your research goals early and often, and set some timelines. I almost never make my own deadlines, and almost always find new other goals that I forgot. But you can get a long way by taking a step back and seeing the big picture in which you are swimming, rather than that one method that hasn't been working for a month. Stupid library preps...
All that is gold does not glitter, and all that glitters is not gold. Corollary: Not all new technology is what it claims to be. Stronger claim: It never is. Stay frosty.
When you are on a roll, rock it. There is no more scant resource then genuine excitement. Push it.
Tons of papers are wrong. Downright wrong. Sometimes scandalously wrong. It's embarrassing, and some of it represents systemic problems with peer review and that whole mess. You can moan about it for a long time (I did). Not sure if that's worth the bellyache, but I'll keep you posted.
Don't make second-hand goals that stake your success on the assessment of the Nature/Science/NEJM intelligentsia, or the ivy pillars of the academe. Those gals and guys have their own little world, and it should come as no small surprise that the people in that club house aren't always all that fun to play with anyway.
Say something, or suck it up. I love to whine. It's a problem that I have. I'm still waiting for an example of when it has helped me. If you have problems with something that's going on in the lab, nip it in the bud and have a conversation. You might learn something, you might stop the problem, or you might at least feel better for having it off your chest. Otherwise suck it up.
The goal is knowledge. The goal is not a first author publication. Find the pleasure in solving the puzzles and exploring the science; it is its own joy. The other rewards may find their way to you somehow or another. Remember what you are building, and why you are building it. Remember your ideal.
Mentorship is everywhere, always take the opportunity to meet people doing different things in different labs. The guy down the hall may solve the problem that's been killing you all month. That actually happened to me today, and it was great.
Look out for #1, a little bit. Cognizant of the goal being science, remember that not everyone has your interests at heart. Example: you are very cheap labor for your PI and he/she isn't itching to see you leave. This cuts both ways, remember that you might not be respecting someone else's need to look out for #1. Example: Your PI has duties besides being on-call for you 24/7.
Related Fact: When your PI says it's easy, it's probably hard. If he says it's doable, it'll probably consume all of your attention for as long as you want to work on it. Remember that you are missing years of experience on the person you're talking to.
Write up your research goals early and often, and set some timelines. I almost never make my own deadlines, and almost always find new other goals that I forgot. But you can get a long way by taking a step back and seeing the big picture in which you are swimming, rather than that one method that hasn't been working for a month. Stupid library preps...
All that is gold does not glitter, and all that glitters is not gold. Corollary: Not all new technology is what it claims to be. Stronger claim: It never is. Stay frosty.
When you are on a roll, rock it. There is no more scant resource then genuine excitement. Push it.
Tons of papers are wrong. Downright wrong. Sometimes scandalously wrong. It's embarrassing, and some of it represents systemic problems with peer review and that whole mess. You can moan about it for a long time (I did). Not sure if that's worth the bellyache, but I'll keep you posted.
Don't make second-hand goals that stake your success on the assessment of the Nature/Science/NEJM intelligentsia, or the ivy pillars of the academe. Those gals and guys have their own little world, and it should come as no small surprise that the people in that club house aren't always all that fun to play with anyway.
Say something, or suck it up. I love to whine. It's a problem that I have. I'm still waiting for an example of when it has helped me. If you have problems with something that's going on in the lab, nip it in the bud and have a conversation. You might learn something, you might stop the problem, or you might at least feel better for having it off your chest. Otherwise suck it up.
The goal is knowledge. The goal is not a first author publication. Find the pleasure in solving the puzzles and exploring the science; it is its own joy. The other rewards may find their way to you somehow or another. Remember what you are building, and why you are building it. Remember your ideal.
Subscribe to:
Posts (Atom)