NEW YEAR PROMO: Use code "NEWDIET" for 20% off a DNA diet test kit Shop Now
Published on

#25 – Studying COVID-19, Nutritional Epidemiology, and the Future of Randomized Controlled Trials with Matthew Amsden

In this episode of the Gene Food podcast, we speak with Matthew Amsden, founder of Proof Pilot. Proof Pilot is on a mission to make it easier to design, launch and participate in studies. Matthew shares with us the design of 3 coronavirus studies Proof Pilot is currently involved with, and we discuss the state of nutrition research later in the episode.

Listen on Google Play Music


This Episode Covers:

  • What is Proof Pilot? [7:50];
  • COVID-19 antibody and mask studies [18:30];
  • Problems with antibody testing [26:00];
  • The future of nutrition science [37:24];
  • Should we trust supplement studies? [43:00];


Matthew: I know it’s really important to say at this point that nobody has all the answers related to antibodies at this point. Just because you test positive for antibodies does not mean that you actually are testing positive for COVID-19 antibodies. There are questions about that, how efficacious these tests are and whether they can differentiate between other COVID viruses that you might have had recently. There are a lot of questions that need to be answered before we can make decisions to say, “Okay, we’ve got positive antibodies so we can re-enter society.” That’s not what the point of these studies are.

John: Welcome to the ”Gene Food” podcast. I’m your host, John O’Connor. Hey, guys. Today we have on a guest who is a startup founder, a former management consultant, and someone who is on the front lines of the research that we are all interested in, both from a coronavirus standpoint as well as from a nutrition standpoint. His name is Matthew Amsden. He’s the founder of ProofPilot. ProofPilot is trying to democratize the process of creating randomized clinical trials in this country. The normal situation these days is that you have to have tens of millions or sometimes even hundreds of millions of dollars and be a pharmaceutical, a large pharmaceutical player to conduct a study, which is why we see most of the RCTs that are out there are on the subject of drugs. And for those of us that are interested in nutrition, we’re kind of left debating the statin trials as proxy markers for how dangerous LDL cholesterol is in the nutrition world.

Matthew is trying to find a better way. He is trying to make it easier and to facilitate studies across all sorts of different verticals from the supplement world to coronavirus. We get into some interesting stuff on coronavirus. He has…his company is facilitating studies for three different topics surrounding COVID-19, one of which is the mental health impact of coronavirus, another of which is the efficacy of the masks in stopping the spread. And the third is what do we need to do testing wise to get these antibody tests everybody is so anxious to get out to the public out there? What do we need to do testing wise to make sure that they’re safe, accurate, can be approved by the FDA, and that they’re not mistakenly flagging for other coronaviruses such as the seasonal flu versus the one that we’re most concerned with, which is COVID-19?

At the end of the episode, we also get into the nutrition world. We touch on high-fat diets, ketogenic diets, body mass index studies that they have going on. It’s a really interesting conversation from somebody who is looking at a lot of these debates from a slightly different angle than we are, which is how do we stop some of the conjecture and start creating more studies that are valuable to people that can guide decisions, from everything from pandemics to supplements to diet. So I think you’re gonna like this episode. Without further ado, here’s Matthew Amsden, founder of ProofPilot. Matt, thanks for joining. We got Matt here from ProofPilot.

Matthew: Hey, thank you for having me.

John: I’m glad we were able to get this interview and record it and get it going. We’ve had some weather and some pandemic issues that have kind of jumped in the middle of this.

Matthew: We have. Just a small pandemic. Nothing serious.

John: Just a small global pandemic that’s just killing thousands of people a day. It’s not a big deal. There’s no need to get too worried. So you are in Maine?

Matthew: I am, yes. I grew up here, so I’m based in New York City, but a couple of weeks ago I was traveling around and I had something to do in Maine in late February. So I came up here and as things were starting to get a little bit crazy, I opened up the summer house, what, three months early, along with just about everyone else in this neighborhood of summer houses. So yeah, I’m here.

John: Interesting. So it’s kind of a lake community in Maine. People come up there seasonally and everybody said, “Screw it. We’re getting out of the city or Boston or whatever,” and…

Matthew: You got it. Yeah. Yeah. I don’t actually know… There are more people here than like the 4th of July weekend it seems, which is very odd because there are no leaves on the trees. So you can actually see the lights on in folks’ houses that you generally in the summertime can’t.

John: And what’s the mood like up there?

Matthew: I think it’s pretty much kind of like this is a big adventure at the moment. This is, you know, it does feel very disconnected. We are technically in a metropolitan area though it’s very much on the edge of the Bangor metropolitan area, which there has been some identified community transmission, but it’s a fairly self-supporting community. And, you know, Mainers tend to be fairly, you know, “We’re just gonna push through,” it because the folks here have to do that every winter anyway. So I think the attitude is pretty good. Now, we see folks walking around on the street, but, of course, you can’t get too close to them. It’s a little early to go swimming and to go voting at this point. So we don’t have quite the interaction that we might in the summertime even if there was a situation where we were living under normal circumstances.

John: Yeah. So the stoicism of Maine is kind of winning the day. There’s not a ton of cases there. Yeah.

Matthew: Not a ton. But there is this community transmission, but so far, not a ton. Yeah.

John: That’s how it feels here. I’m in Wyoming and, you know, I could say, “Well, I’m based in New York City too,” but the longer I spend here, I feel like I’m based in Wyoming I’ve been here so long. I was out of the city. We spoke on, you know, the pre-call and decided not to go back. I didn’t flee the city and come to Wyoming. I was basically out and about and decided, you know, things are getting kind of bad and I don’t wanna go back, but it’s gonna be really interesting to see how things play out.

There was a…I saw a estimate from I think the head of biotech at Morgan Stanley who gave a kind of a loose timeline for when we could expect to return to normal. I think it was targeted at New York City as kind of the most extreme case. And it looked like they were estimating that it could be, you know, June, July before things kind of get back on track.

Matthew: Yeah. I spoke to an economist yesterday and I obviously speak to health experts all day long. And what’s very interesting about the situation is everyone’s kind of making these kind of, not kind of. They are making these predictions. And at the end of the prediction, they always kind of say, “But there’s no precedence for what’s going on now. None whatsoever. So all of us are essentially kind of flying blind,” which I think has created a great amount of innovation and a lot of energy towards health and energy that when we do return to some kind of normalcy, I hope will be applied to other sectors. But I still don’t see a specific answer yet or a specific solution that’s going to give that kind of that date where we all say, “Okay, it’s gonna come back to normalcy at this point.” We’re very much at a period where everyone’s trying to figure it out at a speed that I think is absolutely incredible. But it’s still is going to require a lot of revision and continued conversation to get to a point where we’ve got some answers.

John: Absolutely, and it’s a perfect segue because your startup, ProofPilot, is playing a role in getting us the data that we need in order to return to normalcy. That’s one of the things I think is so interesting about you and I chatting is, you know, your startup, which we’re gonna get to in a minute. I wanna give the audience an overview, but you guys are actually getting an opportunity to kind of be on the front lines of answering questions for people like, “What works better, medical masks N95 masks?” I emailed you that JAMA study last night that I’m sure you’ve probably taken a look at. But before we get into kind of the weeds and the good stuff about what you’re working on right now to kind of bring information out to the public, give us the overview of what ProofPilot is and how you started it.

Matthew: Yeah, so ProofPilot is the platform to design, launch, manage, and participate in clinical trials. When we think of clinical trials, we often think of pharmaceutical drugs. But clinical trials actually apply to a much broader set of research questions than just a pharmaceutical biological agent. We actually set out to create ProofPilot to look at behavioral interventions and social service programs and health and wellness tools because those particular solutions, they don’t have quite as much, not quite as much. They don’t have as much evidence about what works and what doesn’t as something that requires an FDA clinical trial in order to be approved.

So our goal at ProofPilot is essentially to democratize these techniques that have previously only been available to extremely well-funded academic and pharmaceutical trials to a much broader set of research questions. I started ProofPilot out of frustration with the current environment. So about 10 years ago, I was working at a big public policy research and consulting firm in which I was doing research dissemination. And by research dissemination, I mean I was taking the big 300-page reports about what works in healthcare, what evaluation program is actually going to create answers and education, what healthcare program is going to produce the best return on investment, taking little snippets of these 300-page reports that generally nobody would read and use and turn them into things that would actually be applicable and consumable by folks on the ground who were implementing programs or making health policy decisions.

At that particular point 10 years ago, doing things like distributing this material in blogs or email newsletters was really innovative, unique, and fun. So researchers were starting to notice that we were doing this research dissemination at about the same time most Western governments identified HIV as a solvable issue. We know what causes HIV. We know how to prevent HIV. So if we just change people’s behaviors, I know a very naive view at this particular 10 years looking forward. But at that particular point, a lot of researchers figured if we can change people’s behaviors, then we can reduce or eliminate the HIV pandemic around the world.

What was unique about the HIV efforts within research is that many of these researchers were trying to apply the same techniques that they used to recruit and engage participants in a clinical trial as they did in a cancer trial or a diabetes trial. The thing is most of the high-risk populations for HIV tend to be technology early adopters. They tend to be very young. They see no incentive in being involved in a research study. They are the 25-year-old kids who see themselves as invincible. There may be some stigma associated to going into a clinic and receiving treatments or prevention education. So the techniques that researchers were using were the same techniques that they had used for the past several decades to recruit and engage individuals in these studies.

So a couple of thought for researchers came to me and said, ”We know that you’re doing this research dissemination work online. Can you also recruit some individuals for us to be part of these studies?” We shrugged our shoulders and said, ”Sure.” And then with some success there they came to us and said, ”You know, you’re recruiting these individuals online, which is very successful. Can you engage some participants in collecting some data for us?” We shrugged our shoulders and said, “Sure.”

Fast forward to 2011, 2012 we were designing and implementing large-scale research infrastructure for what at that point was considered a decentralized or remote trial in which participants were not going to a specific brick and mortar location. They were engaging entirely online and they were being recruited into that online engagement also online. The problem with all of this was, and I don’t think I’m being overdramatic when I say this, every single one of these research efforts was a complete unmitigated management disaster.

It takes a year or more for a traditional research study done with traditional study sites to be designed and launched. All we were doing at the precursor to ProofPilot was adding on a technology development process on top of that orientation. So all of these researchers would give us a big Microsoft Word protocol document after years or many months, at least sometimes years of back and forth and what that protocol would look like. And we’d say, ”Great, we will have a research infrastructure set up for this within three to six months.” About two weeks before the study was about ready to launch, we would say, ”Here it is. Give us your feedback, and we’ll make little changes.” And some researcher would say, ”Oh wait, I’ve got a new idea. I’m just gonna change it in that Microsoft Word protocol document. And can you make that change in the research infrastructure?”

And oftentimes, those changes would take only a second to change in some plan but would take many months and have a cascade of related issues in the research infrastructure. We’d really rushed to make those changes, to put those changes in place, but inevitably we would push the changes out and they wouldn’t be quite as tested as we would like them to be. Along with that, the studies were completely oriented around the researcher, not the participants. There were a whole bunch of inappropriately applied regulations and inconsistently applied rules. And so it made success really difficult. So with every one of these new efforts, we’d look for a solution out there and there wasn’t one. And that’s why we created ProofPilot.

John: Right. So you have these drug trials, which is kind of the paradigm for doing randomized clinical trials, which is the gold standard of evidence that people are looking for to, you know, you can see it with the current COVID-19 conversation. There’s a lot of controversy about do we have randomized controlled trials that prove efficacy for hydroxychloroquine and Zpacks for people that COVID-19. The problem, what I hear you saying, is that these drug trials are incredibly expensive. I think they can be in the tens of millions and even hundreds of millions of dollars to pull off for FDA approval.

Matthew: That’s the average. Yeah.

John: The average cost. And you take that model and you wanna look at something like a modern problem that’s plaguing, you know, our cities, the HIV outbreak and there’s not the solution there. So basically when you and I had this call, it clicked for me when I first heard about this model, I was concerned. I was like, “Well, I’m like, you know, it’s great to democratize studies,” but I’m like, “What I don’t wanna see happen is I don’t wanna see like brands put together, you know, ‘studies’ and then use them as commercials.” And then I talked to you and it totally clicked for me because you’re like, “No, no, no. First of all, we have a review board that secures against that. But what we’re doing is we’re actually enabling studies to take place by making them more nimble.”

Matthew: Exactly.

John: And that’s what I heard you saying and when I heard that I was like, “Okay, that I get.”

Matthew: Yup, yup, yup. So every study on ProofPilot is reviewed twice as the study is being created. So the first review is internally here at ProofPilot. So we have made very clear that ProofPilot is not an advertising platform. Now, what brands do use ProofPilot to engage with customers and to build credibility, but it’s not an alternative for an Instagram ad. It is a different credible, highly-structured way to engage with a new set of participants in a research study. Those participants could be new or existing customers, but this is not an advertisement.

And if we see an organization going down that path, we very quickly help them course-correct. The second set of review is something called an Institutional Review Board in the United States. In the European Union, it’s called an Ethics Review Process and this is a third-party review. Every study on ProofPilot without fail goes through this process. And they ensure participants are being treated with respect, that privacy and confidentiality is being adhered to, and that the risks and benefits are equitable to the result, to the potential benefits of the study. We’re not engaging individuals in a prison-based setting because they can’t do anything else and they feel pressured to be part of the study. That kind of thing would never be acceptable. And a third-party review board is going to review the study to make sure that that’s the case. And again, without exception, all studies on ProofPilot go through both of those review processes.

John: Right. So when we spoke, we were talking about one of your clients or one of the users of your software on the conducting study site is Harvard University. And we were talking about why would Harvard use this software. And the answer that you gave, which was a fantastic one, is look, we have this infrastructure in place. I’m assuming it even looks like… Do you have user accounts for individuals?

Matthew: Yeah.

John: So you can get people signing up on the study side. You wanna conduct a study and then you have people that are signing up I guess on the, you could call it for lack of a better term, consumer-facing side?

Matthew: Of course.

John: And you have this army of people kind of ready to go so that if an organization wants to come in, you have demographics that have have been created that you can kind of tap when that…

Matthew: We don’t… so what you’re talking about there is something called a participant panel and there are a group of individuals on ProofPilot that can and do join multiple studies. However, ProofPilot is not a recruitment organization nor do we provide a panel of participants because that has the potential to bias research study results. If you’ve got something called a professional study participant, that individual is not necessarily somebody that is going to represent the broader world because most people are not professional study participants.

So we encourage everyone to think about the population that they are trying to engage within the research studies and think about unique, novel, and inexpensive ways to reach them beyond just doing so on the ProofPilot…within the ProofPilot user base because that user base, those are folks who…they’re a little bit biased because they’ve done research studies before.

John: Right. We’re gonna talk about that on the nutrition side here in a minute, which I wanna get into. Before we do, I think people are people are obviously obsessed with COVID-19. We led off with it. Everybody’s thinking about it. We don’t wanna devote the whole show to it. But you did say that ProofPilot is doing a study on the efficacy of face masks, I believe you said, and then you’re doing stuff on the antibody side. Those are two topics that our audience is really interested in. So could you speak to, you know, we have the idea that this is a software platform. What do these studies look like buts and bolts for the COVID-19 front?

Matthew: So first, I must say before introducing these particular studies, is that they are two potential studies within a large portfolio of studies that are going on around the world at the moment. So anything that I say from this particular point, because this particular space is changing and adjusting so quickly, in two or three days, what I talk about here may change quite significantly because of that. So I just wanna put that out there first. So we are looking at several studies at the moment. And I can’t get into the details because we don’t wanna bias those studies if any of your listeners actually engage in them where we’re looking at the impacts and efficacy of at-home antibody tests.

So right now, all of the antibody tests that are available are only available to be conducted within a medical setting. Going back to some of those HIV studies, some of those first big studies that we did were actually efficacy studies and acceptability studies for the at-home HIV test. Those same protocols, those same models have the potential to be applied to the COVID-19 antibody tests. Do these tests… First off, will participants do these tests at home? Can participants understand the results that they get from these tests? And if they do understand the results of these tests, does it change their behavior? Does it make them less anxious?

So those antibody tests, we are continually speaking with the FDA, with local and state regulators to make sure that we were following every regulation that there is. Those studies should be launching, assuming that everything goes as we hope, within the next couple of days after recording this particular podcast. Another set of studies that we’re working on relate to anxiety and COVID-19. These studies are generally with psychiatrists at academic institutions and we are looking at solutions that a participant can do at home to manage their anxiety because it’s not just the disease itself, but it’s all the fear that comes along with it.

And finally, the third group of trials that we’re looking at are different devices to protect mostly healthcare workers from either infecting their patients or patients infecting them. And those are generally mask trials, but there are several others that are in discussions as well.

John: So antibody first. It seems there’s kind of another region there with those antibody tests because I know there are biotech companies that are selling them to physicians. There’s a physician here that I’ve become friends with in Wyoming where I’m kind of riding this whole thing out and he’s ordered 500 of these tests and he’s gonna administer them. I know there was widespread testing done randomly in Telluride, Colorado. I think a local philanthropist ordered essentially a test for the entire San Miguel County region. Where does this sit? So physicians can order them and administer them to patients.

Matthew: So let’s first…

John: What’s the regulatory status?

Matthew: Let’s first differentiate between a test that actually determines whether you are currently infected with COVID-19 and the test that actually identifies whether you have antibodies from a prior infection of COVID-19. So physicians now and hospitals and drive-up testing centers, most of those, not all of them, but most of those at the moment are actually testing for current COVID-19 infection. The studies that we’re doing are looking at antibody tests. Those antibody tests are for those individuals who have previously tested positive for COVID-19 or in for individuals who are pretty sure that they had COVID-19 in the past.

Now, those antibody tests right now, you must be a high-complexity lab in order to distribute these tests. And so most independent physicians do not have access to these tests because they don’t have a high-complexity lab. And for those organizations that do have a high-complexity lab, they must be conducted within a facility and with medical supervision. We are looking at trials to see if those tests have efficacy, have acceptability, and appropriately prepare individuals to change behaviors in an at-home methodology. It can only be done in coordination with a high-complexity lab and it can only be done with all of those ethics reviews and within state and local regulations within a research study at this point.

John: And that’s a great distinction. That’s a great point, that the antibody test is not the active infection test. They’re separate tests. I believe the antibody test is gonna be rolled out as a finger prick.

Matthew: Yes.

John: And the active infection test that was announced by Abbot, the super-rapid ones are still a nose swab that goes inside the nose.

Matthew: That’s right.

Jon: I’m particularly interested in the antibody tests and I’m sure you are as well as New York people that live in New York City, I feel like your life is gonna be… Even though there is some doubt as to whether you can be re-infected, there’s doubt as to how long the antibodies last, all these different types of things, I think it will give people a tremendous peace of mind, especially people that are planning on returning to their apartments in New York City and in other cities. Not just New York City, but New York City is an epicenter of this, tremendous peace of mind to know that you had an infection and you weren’t ravaged by it. You might have gotten very sick, but you survived and you’re okay. I think that’s something people really wanna know. Speak to that.

Matthew: So I think it’s really important. Not, I think, I know it’s really important to say at this point that nobody has all the answers related to antibodies at this point. Just because you test positive for antibodies does not mean that you actually are testing positive for COVID-19 antibodies. There are questions about how efficacious these tests are and whether they can differentiate between other COVID viruses that you might have had recently.

So there are a lot of questions that need to be answered before we can make decisions to say, “Okay, we’ve got positive antibodies, so we can reenter society.” That’s not what the point of these studies are. And anyone who misinterprets the points of these studies will automatically be unenrolled from the study because that’s not the point here.

John: I would be immediately unenrolled from this study. I would right away be unenrolled because [crosstalk 00:25:30].

Matthew: Absolutely. It’s absolutely essential.

John: No, I’m just [inaudible 00:25:34] with you.

Matthew: It’s absolutely essential that folks understand that we are in the discovery phase with all of this stuff at the moment. And to participate in a research study like this, you have to understand that none of these tests have been FDA-approved yet. They all are under a very unique emergency declaration where they’re being allowed to be sold in very defined situations. None of them are allowed to be conducted at home except within a research study. And even if you have those antibodies, it does not mean that you actually had COVID-19. It does not mean that you necessarily have immunity. It just means that you have an antibody for some COVID virus, not necessarily COVID-19.

So we have to run multiple studies asking multiple questions over time to actually get to some of these answers. And jumping the gun and saying, “Oh, I’ve got an antibody,” and going out into the community creates a whole bunch of risk that is really important that everyone understands that’s not what we’re [inaudible 00:26:35].

John: Yeah, that’s a great point. I mean, there’s a solemnity especially to what you’re doing because you guys are running these studies and you have to take this very science-based approach. I think it was for me, I saw David Sinclair at Harvard. I don’t know if you’re familiar with David Sinclaire at Harvard. He’s longevity and head of the genomics department at Harvard. He actually has shared one of the anecdotal stories about some of these antibody tests and he was saying, I believe, that he was able to get some at home and administer them, and it came back negative.

I think that the consensus is that for some of these tests early on, more to know. All these COVID-19 discussions, it’s great that you’re giving that caveat because this is an emerging field and there’s far more question marks than there are answers. But I believe that people are saying early on that they believe that they’re about 90% to 95% effective at finding, you know, coronavirus antibodies in the bloodstream.

Matthew: Right. And another distinction here, COVID-19 is not the only coronavirus. There are multiple others. So if you felt super sick in let’s say November of last year and you had symptoms that seem very similar to a lot of the symptoms that are being tossed out around the COVID-19 today, you may have had another coronavirus. Does that provide you any kind of immunity to the current Coronavirus? That’s anyone’s guess at this particular point. And these are all kinds of questions that we need to answer within a research process.

John: Yeah, I think it was the Seattle flu study that keeps samples of people in the Seattle area that have had severe respiratory infections and flu. And they retrospectively went back and tested those samples recently for COVID-19 starting in January. I believe they did that to explore this theory that there was herd immunity in California. People are theorizing, “Well, why is California not having as severe of a reaction to COVID-19 as say New York City?” There’s a number of theories that are out there. They found that in the Seattle samples that they went back and tried to match to COVID-19 retrospectively that they weren’t getting hits until February, which they were saying, you know, there’s some problems with that, one of which is that 50% of people are asymptomatic. A lot of people don’t have respiratory infections, but yeah, these other coronaviruses are out there. That’s a fantastic point. Go ahead.

Matthew: No, I was just gonna say I think with this, it’s worthwhile discussing the phases of research, given that we’re all focused so much on solutions for COVID-19 now, and there is frustration with the kind of lack of what or what feels like lack of progress. At the same time, there are claims that some malaria drug works and then there are claims somewhere else that the malaria drug doesn’t work. I think it’s worthwhile to discuss the rungs of the research process.

And what we try to do at ProofPilot is dramatically speed up those rungs. But there still is a process. There still is the scientific method. So from ProofPilot’s perspective and mine personally, the first rung of that process is what we call the anecdote. Everyone knows the anecdote. We’re all human. We all have stories about what works and what doesn’t for us. A lot of times, anecdotes get dismissed in the scientific process, particularly when it goes to the public press because it is that. It’s an anecdote. It’s not scientific proof. However, from ProofPilot and my personal perspective, if you don’t have an anecdote, you don’t have anything to study. You don’t have any ideas.

If I’m talking about my blood glucose levels and I wore a blood glucose monitor for a couple of months last year and I saw my blood glucose spike in certain days and not on others, and looking back on it I was thinking, you know, “Those days where my blood glucose levels spiked, those were days that I had a high stress level,” that is an anecdote.” Is there any…was that a research study? No. Is there any scientific proof to that? No. I’m only looking retrospectively back at what I was doing and I have some objective data in that blood glucose monitor. But there is, it is an anecdote. It is an N of one result based on my kind of feelings.

The next step of that process after anecdote is actually to start getting more structure about it. So I have a theory. I have a hypothesis that stress has an impact on blood glucose levels. So maybe we can start pulling together a couple of other people who also look at their blood glucose monitors on Dexcom or Abbott Freestyle or whatever and they also see the same increase in blood glucose levels with days of high stress. Those are case studies. So we now have multiple cases of individuals who are all saying the same thing.

The next step would actually be to get some experts involved, folks who know about stress, who know about blood glucose, and have them review some of this material and kind of say, “Yeah, there’s something going on there.” At that point, you can start doing structured trials. So you actually get folks into trials and start measuring things in a systematic basis across a broad set of individuals. Now, these trials can be single-arm trials, meaning that there’s no placebo, there’s no control. They can be multiple-arm trials where you have some folks that have the blood glucose situation that I’m describing where you’re introducing some stressor to them and some trials where there is no such introduction of any kind of treatment or any kind of situation.

Those trials are each going to create results or create some kind of finding. And the most important thing that I hope anyone takes away from this conversation, one trial that has a result associated with it is not the definitive answer. The scientific method requires that you run these trials multiple times in different situations, different time points, and different environments. And if each trial comes to the same answer each one of those times, then each trial is building on that evidence and making the evidence stronger.

We are in a culture now, particularly at the moment, but this predates COVID-19, where there would be results. It gets published in academic journals because academics, they want a new result, they wanna get published, and then it gets picked up by the broader press. That is is one step in scientific evidence. It is not a de facto answer. It is multiple replications of that same trial that gets to that answer. And in the COVID-19 situation where we have a potential solution, just one trial does not mean that that potential solution is the answer. It means that it has worked its way up that chain, that process of scientific evidence, and now it’s time to replicate that same trial in a different set of situations and environments. And if the same results come, then we know that we can be even more confident that the answers hold up.

John: Yeah. How do you feel about epidemiology in that regard, especially nutritional epidemiology?

Matthew: Yeah, so there’s something called a prospective trial and there’s another called a retrospective trial. So most nutrition research studies are not trials in themselves. They are retrospective analyses where we are asking individuals to remember what they have eaten or what they’ve done or what they have completed in the past. And in many cases, these trials are asking people to remember their diets 30 days or even previous to that. Most individuals, myself included, I don’t remember what I ate last week, much less 30 days ago.

These trials also are not controlled. So we’re asking for folks to remember what’s happened in the past and then we were making a whole bunch of assumptions based on what’s gonna happen in the future. Again, those retrospective trials are really important. They give us some idea of what we might study in a prospective trial. A prospective trial, like a clinical trial, like an observational trial, these are trials in which we are actually asking people to do certain things and track on an ongoing, into the future basis. These trials tend to be more validated because they are more controlled and we are managing what people are doing in that environment.

Retrospective, again, valuable because they give us ideas for things that we can do prospective trials on, but they are not necessarily definitive answers. Retrospective trials are one of the reasons why there’s all this controversy over is an egg healthy or is an egg not healthy? Well, it depends on a lot of things and it also depends on whether folks actually remembered that they ate eggs at a certain period of time. In retrospect to trial, they might not.

John: What they cooked the eggs in, you know, how much cholesterol they’re absorbing, you know, what their microbes state of their microbiome is. We did a whole episode on eggs a few back. That’s a great point you make for people that are like, “Well, retrospective? I kind of still don’t get what he’s talking about.” You have these huge studies like the nurses’ health study and these huge epidemiological studies that are collecting data for purpose A and then there are huge data sets. And as part of that data set, they might’ve had food frequency questionnaires where people kind of lose the estimate of what they’re eating.

Researchers will go back and revisit those old data sets and be like, “Okay, this isn’t what the purpose of the study was originally, but we’re gonna retrospectively go back and parse through this data and see if we can get some nutrition insights from this old data set,” whereas what you’re referring to here is prospectively. We start at the very beginning and we were telling people, “Okay, we wanna study what’s gonna happen to your blood, to your lipid markers if you’re eating 21 eggs a week.” And we’re starting now.” Okay. So people are focused on it because they know that’s the purpose of this study. They’ll be presumably a little more careful.

Matthew: Yeah. Right, right.

John: So how do you see the whole nutrition? I mean, our podcast, our health and wellness websites, we’ve been doing stuff with COVID, but we’re really more of a nutrition web platform. I can see… Erin and I, who’s the head of research and the lead geneticist here, he’s been out. We usually do shows with him, but he’s had a new baby he’s taking care of. He’s talked about how technology like yours could actually be the death of epidemiology because or using technology, iPhones, maybe not saying… I could see you saying maybe not the death. Could you say maybe a total changing in the paradigm of the accuracy of epidemiology?

Matthew: I hope that it progresses epidemiology and I hope that it supports epidemiology. I don’t think epidemiology is ever going to go away in the way that it is conducted now because things like disease surveillance, you can only do disease surveillance retrospectively because it’s not ethical to give somebody a disease.

John: Let’s focus on nutrition. Yeah, right. Of course. Epidemiology as a field stays intact. I’m sorry, I should’ve been more clear. Nutrition epidemiology.

Matthew: Yeah. Again, I still think that there is value in looking backwards, in looking retrospectively because it gives us ideas and gives us the platform to do prospective studies. If we see…let’s take honey, for example. If there is some indication in some retrospective study that’s a certain kind of honey creates some kind of health outcome, that is something that we didn’t know otherwise. We wouldn’t have known to do a prospective study on it.

So I do think those retrospective, that traditional epidemiologic model is still really important because it helps us narrow the field of what we do prospective studies on. I do hope that the orientation of making most of our assumptions based on retrospective studies shifts to focus more on prospective studies. We don’t want to end up in situations like we did in the 1990s and early 2000s where we did retrospective studies and assumed that high-fat diets created heart disease. We now know that that was probably not necessarily the full story. So I hope that we start to be more balanced in doing prospective and retrospective studies so that we can more quickly get to answers that may validate or invalidate some of the assumptions that we’re making from those retrospective epidemiological studies.

John: Sure. Okay. That’s a great answer. That’s an interesting point you bring up about the idea of fat and heart disease. That’s something that we’re very nerded out on here at Gene Food, very interested in. When we spoke previously, I know you said that you were doing some work with some researchers at University of California San Francisco. UCF is sort of one of the leading thought leaders. The researchers there, some of the doctors there are some of the thought leaders in this space that is rethinking and revisiting the attitudes that we’ve had towards dietary fat.

I will say just as an aside, I do think for certain types of people, it’s very clear that high-fat diets actually do cause heart disease. But I think that it’s clearly been overstated for a lot of people. I’m fascinated to know what kind of studies are you doing on this nutrition side, on the dietary fat, anything like that you can tell us about that?

Matthew: Sure. So we work with a number of supplement companies at the moment and they do research studies in all various different kinds of outcome orientations. We have one running now that’s actually in skin health. It’s called the DermAid study. It’s a large nationwide study in which individuals are enrolled in the study and provided with various different formulations of dietary supplements. They take a selfie at baseline, at 45 day, and 90 day. And dermatologists around the world are actually assessing individuals’ facial skin to see whether or not these dietary supplements actually improve their facial skin health. So that’s one.

John: Are these collagen supplements then or something else?

Matthew: I can’t tell you exactly what the items are because it would bias the results, but collagen is one of the components. And there are various others. Another dietary study that is open at the moment is with Life-Span, which is a Swedish diet and lifestyle brands. The study is enrolling individuals in the United States and is looking at a traditional diet process and how it changes essentially well-being and mental health. So do you get angry when you’re on the diets? Do you over time start to feel better? Do you change your attitudes about different kinds of foods while you’re being on a diet?

These are questions that a digital health company took it upon themselves to answer because they weren’t necessarily seeing the kinds of answers in the traditional academic press that they were hoping for. So that’s another. We have done a number of studies with the Snap-Ed program in the United States. The Snap-Ed program for those who are not familiar is part of what used to be called the food stamp program. So those individuals who don’t necessarily have the resources to purchase food, they’re given, SNAP supplemental nutrition assistance program credits where they can use them in a grocery store.

The unfortunate thing today is that until very recently, SNAP benefits couldn’t be used to redeem and pay for groceries online. So between 14% and 17% of individuals in the United States are on SNAP on any given day. And those individuals can’t purchase groceries online until fairly recently in most states. So we were running studies to actually see how purchasing groceries online changed nutrition behaviors and general health outcomes. So we don’t limit ourselves to this particular dietary supplement has this particular health outcome. We believe very strongly that every research study and every health outcome has various components to it. There’s a behavioral component, there’s an environmental component, there’s genetic components, and then there is a kind of a physical health component. And in many of the research studies at ProofPilot, we’re looking at all four at the same time.

John: Yeah, the dietary supplement world of studies is interesting. As a consumer, I do assign a greater degree of trust to companies that take the time to go and do studies. I have also seen, especially in the nootropic space, you have a nootropic company that will say, “Well, we’ve proven that our nootropic increases alpha brainwaves in such and such a way,” when really there’s already been a pretty well-established body of evidence that alphamine does that standing alone. And so they’re pretty much just piggybacking on like the alphamine research and then saying, “Oh, our supplement does this too. And oh, guess what? Our supplement also has 50 milligrams per serving of alphamine,” as did five other studies that showed that alphamine standing alone aside from any nootropic formula or branding already increased alpha brainwave. So there’s the trust factor. And then there’s also like the really getting an understanding how this data is packaged

Matthew: Yes. So what I see a lot of supplement companies doing right now, it’s an extremely competitive space. And so you have Vitamin C and how many different components or different formulations. Increasingly to get a leg up, many organizations are doing these prospective studies so they can actually show that their particular formulation that has Vitamin C plus maybe five or six other things in it actually gets them that little bit farther ahead than an organization that has a slightly different formulation.

So we’re seeing a lot of those studies and we expect to see more of them as all of this kind of economic and cultural upheaval that we’re in at the moment starts to affect the dietary supplement space from an economic perspective. We’ve seen many more of those organizations say, “We’ve got to do something else other than put up an ad on Instagram. We’ve gotta actually show that our unique formulation is a little bit different and a little bit better.”

John: It sets them apart. And I think substantively so. I think so. So closing out, we touched on a minute ago, it’s a big topic for us. We kind of flirted with it a little bit, but do you have any cardiologists? I know you can’t sometimes get into the exact details and that’s fine. Tell us what you can. If you can’t tell us about it, don’t tell us about it. But are there any cardiologists that are planning studies doing any kind of… We love the RCT nutrition studies. They’re so few and far between and the ones that we get, we’re like, “Oh, yes.” And so do you have anything like that planned?

Matthew: We do have several of them. I can’t give you any details at the moment again, because we don’t wanna bias the results. I know it’s hard. It’s hard. But we do have some coming up that are looking at various different kinds of ketogenic diets. So there’s the dirty keto and there’s keto and the Mediterranean keto and vegetarian ketos. So we do have several coming up that are looking at the various outcomes of those different kinds of ketogenic diets among individuals who are at extremely high risk of heart disease and other heart complications.

So those studies should launch, I would expect, within the next two or three weeks although, you know, some of these studies do require some heavy-duty blood work that require individuals to go into a clinic to complete that blood work. So that may delay as we’re all dealing with a social distancing. We have several studies for athletes as soon as individuals can go back into clinics that will look at heart health and heart strength as it relates to different recovery modalities and different recovery tools. So that’s something that we’re looking forward to as well.

John: Cool. I definitely wanna hear about these. So you’re taking patients. It sounds like some of these, I’m assuming cardiologists, are researchers at some of the universities you’re working with are gonna take patients with advanced heart disease and put them in ketogenic diets.

Matthew: Typically not with advanced heart disease, but with some very high BMI levels.

John: I got you. Okay. I got you. Okay. That’s kind of a weight loss protocol. Interesting

Matthew: Weight loss, but there’s also other components to it as well. And I wish I could be more specific, but again, we don’t want to say exactly what the research studies are and then have people join it and already know what they want their outcomes to be. So we have to be very careful in how we present these studies so we don’t bias. And we are getting people in that are coming in with an open mind. It’s a little bit like being a juror, right? You need to be unbiased when you come in and sit down.

John: I’m thinking maybe just with these, and don’t comment, but I’m thinking maybe people that have high BMI, maybe they have very triglyceride-rich LDL particle, they’re insulin resistant, they have a lot of, you know, high VLDL, high small dense LDL. And maybe we’ll see what those interventions do to that ”phenotype,” which is really exciting. I can’t wait to hear. Once those go public, please do tell us and we’d love to absolutely love to hear more about those.

Matthew: And anyone who wants to learn more about what studies are available at the moment, they can go to That’s P-R-O-O-F-P-I-L-O-T .com. And all of the highlighted studies that are currently enrolling a general population are available right there on that page. If you find a study there that you’re interested in and for one reason or another you’re not eligible, which is very common, other studies that may have a more specific and narrow eligibility requirements that are more specific to you will be available for you within the ProofPilot app. And you don’t need to download anything. It’s all just on the web. You can use your mobile phone or you can go on your desktop computer. It doesn’t matter

John: Yep. And we’re gonna link to the site on the show notes as well. In closing out, it’s been a great, I think a very good, very wide-ranging interview. I’m really glad to have had you on. Let’s close out with your crystal ball prediction for where all this COVID-19 stuff is gonna go. What do you see happening for New York the next few months?

Matthew: As opposed to trying to predict the future, I wanna say what I hope the future is. And that goes beyond just coming back to some level of normalcy. Obviously, I hope that’s the case. I hope it goes beyond that though. Where we are right now in the COVID-19 pandemic is obviously very serious. But as we look across our healthcare ecosystem at the moment, there are 300,000 people in the United States who will be diagnosed with Lyme disease between probably now and July. There are 36, 38 million people who have diabetes. There are 150,000 individuals in the United States that have HIV and don’t know it. There are, what, a third of the population has hypertension. These are all issues that affect our public health infrastructure. They are in many cases issues that actually increase the mortality and morbidity of COVID-19.

I hope all of the energy, the innovation, the regulatory changes that we’re putting in place for COVID-19 because it’s such an acute issue right now, we start to think about how those techniques can be used to improve public health and address issues across all of these other topic areas. I also hope that we look at not just addressing issues and curing disease, but also think about how we can use things like nutrition and meditation and other modalities to improve health, improve longevity, and create a more healthy environment as opposed to just saying, “Okay, COVID-19’s done. Everything’s back to normal,” because things were not necessarily perfect beforehand. And I hope we use this opportunity to address a broader set of questions.

John: Awesome answer. Awesome answer. Thank you for your time, Matt. Congratulations on your startup. I think you guys are doing great work. I really appreciate you taking the time to speak with us and stay safe.

Matthew: All right. Thank you.

John: Yep. Bye-bye. The ”Gene Food” podcast is our attempt to synthesize the latest developments in the fields of genetics, nutrition, and medicine, and offer you practical tips and stories you can use in your own unique health journey. If you enjoy this podcast, you can find more information online at

John O'Connor

John O'Connor is the founder of Gene Food, a nutrigenomic startup helping people all over the world personalize nutrition. John is the host of the Gene Food Podcast and a health coach trained at Duke's Integrative Medicine Program. Read his full bio here.

The very latest on genetics, nutrition and supplements delivered to your inbox!

Facebook icon Twitter icon Instagram icon Pinterest icon Google+ icon YouTube icon LinkedIn icon Contact icon Info icon