Clinical Trial Expert Discussion: Study Design for Microbiome Therapies

Human clinical trials within the functional foods and nutraceuticals industry tend to follow a consistent flow, even though aspects of the process are relatively flexible.

However, failure to plan & manage a clinical study effectively can result in large overhead costs and missed deadlines. Several core variables must be considered to ensure a successful study, including budget, study duration, and number of participants. Timelines tend to vary depending on the health area, intervention period, and number of samples to be analysed.

Atlantia Clinical Trials recently hosted a roundtable discussion, gathering some of the companies leading experts to discuss an array of common problems and issues around the conduct of successful food clinical trials.

These experts included Suszie Tyree (Research Manager), Gillian Dunngalvin (Statistician), Onthatile Serehete (Study Doctor), and Alice Eggleston (Physician Assistant).

1. ‘What is the normal clinical trial process?

Suszie Tyree (Scientific Officer)

Once the study has been agreed upon/the deal has been closed, the research projects start with my team. The first step is to complete the protocol development, which is done by putting together what the main aims of the study are. One of the first steps we have is defining our objectives, which is our main goals of the trial. This is when the measurements are specified, for example, the effect of 12 weeks of treatment on a probiotic on stress, and then for that objective we would define the end point; or how we’re going to measure that because we want to know what assessment tools we’re using. For stress we would run through some possibilities i.e. Cohen’s precision scale, and we would define what an improvement would mean for that, where we would want to see a decrease in Cohen’s perceived stress score over that 12 weeks and usually we have a couple of different objectives in the trial, so we would normally have a primary which is our main one, the one we care most about and then you can have some secondary objectives as well.

Closely associated with stress is often sleep, therefore you might want to put in a sleep assessment as the secondary and then if there are variables that are being looked at where it’s not explicitly known what direction they’re going to go, its common to put those as exploratory. For example, we could put in the effect on the microbiome over 12 weeks and so we’re not sure how the different bacterium groups are going to change but we want to see it, so we would add that in as an exploratory endpoint, and so that is defining our objectives and endpoints.

The next step is to define the population we want to study, which is quite important to figure out, whether you’re looking at a healthy population or a diseased population. You would define that in your inclusion-exclusion criteria to ensure that you’re looking at the correct people, then we’d get into the visit by visit running it down, what we’re going to do with our participants, when the assessments are going to fall, screening visits, baseline visits, sort of just figuring out what the trial is going to look like.

Alice Eggleston (Physician Assistant)

My role as a physician assistant in the Chicago site of Atlantia, I help oversee the implementation of the clinical trial, so after we get the handoff from the protocol development team, it then comes to us and we have a site of research coordinators and clinical trial assistants and phlebotomists. We all work together along with the project manager and operations manager to ensure that the study flows smoothly. We have recruitment team who bring us the participants, we do a screening visit to assess that they’re the fit within the protocol requirements and then depending on the protocol, we’ll follow along for a couple weeks or months and provide them the intervention. Whether it’s a probiotic or we’re doing oligosaccharides, so different investigational products.

The participants will have their blood drawn, they’ll get questionnaires throughout the study, so depending on what is involved in the protocol and what the sponsor is looking to address, we are the ones that implement all of that and collect all the data we observe for safety and tolerability.

We then collect their responses on different questionnaires, whether about signs and symptoms or about quality of life or mood. Then data is then entered to an electronic case report form. Once we have all our participants and they’ve gone through all their visits we would do a close-out visit. Throughout the trial the study is monitored by a clinical research associate who ensures that we’re following all the steps that we need to make sure that we’re reporting correctly, and we work closely with the principal investigator for any issues along the way.

 Onthatile Serehete (Medical Doctor)

As clinicians we maintain patient safety, we work very closely with the operations team to ensure that everything goes well and at the end we review if there’s been any clinical problems and once the data has been collated and monitored to ensure that it is reliable data for the sponsor, we have a close out and everything is handed to our statistician, who handles the stats of course.

Gillian Dunnglavin (Biostatistician)

At the end of the trial, once the last participants last visit has been complete, we move on to the cleaning of the data ensuring that it’s valid and accurate and has high quality. Included in that we’ll allocate the participants into populations, so those populations can be the intention to treat or also known as the ‘full analysis set’ and the best way to view that population is your real-world population.

They may have not taken every single dose of the product, they may have gotten ill during the trial and had to take an antibiotic, and we decide on how those medications and those different events that occur during the trial will impact both safety and efficacy.

We make that decision during what’s called a ‘blinded data review’ that has a committee, and that committee is made up of a statistician, a clinician, a principal investigator as well as a sponsor representative. Once we’re happy with the allocation of our intention to treat we also look at our ‘per protocol’, and a best way to view that is our perfect population.

They didn’t take anything that they weren’t meant to, they took all the product as they were needed, so that’s the population that’s a ‘per protocol’. It’s important to know when we’re allocating people to these two different populations that the intention to treat or the full analysis set is most valuable to the regulatory bodies as well to publication standpoint, and the reason for that is the generalizability of results. If we look at someone taking a product in the real world, they’re not going to take it every day, they’re not going to take it every single morning on a fasted stomach as you require them to do it, they’ll take it on a more ad hoc nature, which is human nature so to take that into account that’s why regulatory bodies want to see that your product works in that intention to treat or that real world population. Once we’ve allocated people into the populations and we decided it’s clean and we’re good to go we proceed to database lock on blinding, and we go on to the analysis phase of the trial. This is where we run statistics and then we come back to the clinical team and our pi, who give the interpretation for the results.”

2.“What are the microbiome endpoints that you usually look at?”

(Suszie Tyree)

The human microbiome is an ever-evolving field and there’s a lot to be looking at, so this does change over time as we get more information about it, currently our key analyses are looking at things like alpha changes in alpha diversity or beta diversity and seeing any sort of significant changes in the presence of some strains of bacteria over others. There are of course a lot of publicly available databases of microbiome data allowing for some room to be comparing shifts in your population with those that you have seen in publicly available databases.

There are a couple of different ones you can choose from but usually our standards are sort of the alpha beta and differential abundance analyses are the key ones that we are sticking to currently, but it is ever evolving so it could change in six months.

(Gillian Dunnglavin)

“Furthermore, it really depends on what phase your trial is in development, and what objectives are used. If it’s in the exploratory phase and you’re just trying to see is there any benefit of your product in the microbiome, then you might do a deep dive in the bioinformatics just to see what happens in the patterns. However, if your trial is further along in development you may be focused just on the microbiome that is linked to for example fibre and or other products if that was what was in your study. So, it really depends on what phase you’re on, whether you want to do a whole exploratory look and just try different methods and see what kind of trends are interesting to inform your further study development. Because most trials are part of a dossier package where you have four to five trials or more depending on your goal which start from pilot go to exploratory, and then move into confirmatory trials”

3. “When running clinical trials for microbiome-based products would you exclude vegetarians?”

(Onthatile Serehete)

No, we do not exclude vegetarians, we’re looking for real data that affects real people, so we would include all sorts of diets, and this is information that we would collect with demographics. We want to know how old are you? what do you do for a living? do you smoke? how many units of alcohol do you drink? what is your day-to-day diet? because when we analyse the data, before it’s unblinded or even after it’s unblinded we might see trends and then we can look back and see oh was there more positive effect on vegetarians versus pescatarians? So, no we do not exclude vegetarians, we want a normal sample as much as possible. We mostly tend to exclude medications like antibiotics and people taking probiotics and prebiotics just to avoid confounding. It really depends on 1) the investigational product and 2) the type of study that we’re doing. Is it confirmatory? is it exploratory? so all that will contribute to how we decide on what is on the exclusion and exclusion criteria.

(Gillian Dunnglavin)

The only time we would really exclude vegetarians was down to their choice themselves, so if the product happened to have animal products within the investigational product, we would of course disclose that to the participant before they enrol in the trial and they may then make the decision depending. Some vegetarians it depends so they would make the decision then based on their diet whether they want to be in the trial or not, so yes we keep vegetarians in, but we would of course disclose all the ingredients in case there was an animal product within there that would contravene to their preference.”

(Allice Eggleston)

There are some other diets that study participants might be adhering to, to address their clinical symptoms. Take an IBS study as an example, they might be prescribed to adhere to a low fodmap diet. That’s avoidance of certain foods and for instances like that there are certain diets where they would be excluded just because it would be difficult for us to assess; are the benefits because of their strict adherence to that diet or if it’s because of the intervention that we give them. Therefore, it depends on what the sponsor is looking for, and what they’re hoping to learn from their product but there are certain situations where diet can affect the study population.”

 

4. “If you would choose for a healthy test population, how would you measure the true health effect of the intervention?”

 (Onthatile Serehete)

True health effect, there could be clinical significance, which is what myself and the Physician Assistant would assess based on our medical background; blood pressure & vitals could have a statistical difference, and the two may not match as clinically significant, therefore they may not match to a statistical difference. One should bear that in mind when they’re asking about the effect of any intervention on the test population, that there may be health benefits but not statistically making a difference.”

(Gillian Dunnglavin)

“Often times we see in publication the reliance on statistical significance for getting a paper published, p values are influenced by a sample size, so the largest sample size you are the more chance you’re going to find natural variations within a population and you may pick up a p-value or statistical significance. However, you’ll see an increasing desire from a lot of publications for you to also include confidence intervals or effect sizes, and they really show the true strength of that change.

Is it a natural change? is it a small change? medium or large? and then based on that measure of the change, it helps inform clinicians in their interpretation of those results. It’s really important that when you’re looking at publications, what is the true benefit? It could be statistically valid but is it also clinically meaningful? and you’ll link to that terms such as minimal clinical important difference, responder status, and we’re seeing more and more publications using these on a regular basis.

I think the question wanted to flag to you ‘how can you measure a change over time in a healthy population?’ and it’s just using sensitive tools, sometimes we may have a healthy population but require them to be in the lower range within the healthy parameters so that there is still room for improvement. We would also use sensitive questionnaires and measures to be able to pick up on that, that’s how we measure it within a healthy population, it can also help with recruitment if you focus on a healthy population, particularly for your initial studies, and then once you narrow down what exactly the effect of your product is, of course move to either a population that you know subclinical or a clinical population and then test your product but for those initial phases I think it’s always good just to test it first in a healthy population using sensitive measures.” 

5. “Why is data important and what are the barriers and solutions to it in trial design?”

(Gillian Dunnglavin)

There’s more and more focus on data, in the past people thought you collected the data, it comes back out you analyse it, that’s it. But there’s so many processes behind the scenes and it’s a shared responsibility from the sponsor, as well as all the clinical trial team, from the person who collects the data to the person who enters it into the data capture system, to the person who then reviews it from an interpretation point of view, and then all the way through our data quality and data management teams.

The reason why data is so important is if we don’t have good data what do we have? we can’t ensure the safety of the participant if we’re not sure if the data is valid, we can’t ensure that our conclusions are valid which is why you’re seeing more and more focus from a regulatory point of view on the terms like data integrity, data quality, and data management methods.

If you review FDA ‘warning letters’ which companies receive, you’ll see that they are often referring to data integrity, data quality, and data management issues. In terms of things like barriers to it and solutions to it, the main ones tend to be lack of understanding, particularly lack of understanding of ICH-GCP or the International Conference of Harmonization for Good Clinical Practice, or FDA requirements and  for European FSA  requirements for what is good practice.

The way to get that out is making sure that you have the training and education that your team needs, if you don’t have those internal resources, outsource to people like Atlantia that have those resources and can help train your teams, and get those resources for you. The other issue that can occur with poor data management and poor data practice is inconsistent practice, it is imperative that every single person collecting the data, is doing it in the exact same way. From the way they word the questions to how they input the data, even a decimal point of difference.

Let’s say one person inputting a value to one decimal place, and another person inputting it to two decimal places, that makes all the difference for finding those small changes over time, so that’s another issue.

The other one we see a lot that you’ll see flagged are not having a statistical analysis plan, though it might seem like a very meaty document, and you may ask “why you need a 50-page document to explain this or 30 pages?”. It really outlines everything you’re going to do, and it’s decided before the database is locked, one of the issues that often comes up that is a criticism when you go to publish or if you go to a regulatory body, is that they will say “it looks like you made the decisions for the analysis on an ad hoc basis as you were going on”.

They could argue that you were deep diving, or you were making decisions based on what looked best for the data, you excluded that person because it looked better to take them out, that’s why we at Atlantia have set procedures in place that ensure we review the data blinded so there is no influence based on what product they’re taking, and then we lock the database. Before database lock we have a clear plan that says if we have outliers, we’ll use this method and outliers are simply an extreme value. It also outlines how we handle missing data, how we’ll present the results, what the tables will look like, and it also makes it much easier to review at the end. They are the main barriers and the solutions to why data is so important.

 

6. What is the general size of the trials involving microbiome therapies versus clinical support tools?

(Gillian Dunnglavin)

It’s extremely difficult to give an exact figure as it really depends on what phase the trial is in, and the goal of it. We talked a little bit a moment ago about clinical significance versus effect sizes, if you have a small effect size that means you expect a really small change, but that small change isn’t clinically significant, you’re going to need a much bigger sample. That’s when you see the trials that have 100, 200, 300 per group, the smaller the change you’re expecting over time, the more participants you need. If you’re expecting a really large change over time, that would lead to a larger effect size which is a more obvious clinical significance, then you could have smaller sample sizes.

In general, for a pilot study I wouldn’t really drop below 10 to 15 participants per arm, and the reason for that is just in case of dropout. Furthermore, without that there’s not much I can tell you from a statistical point of view, I will be at best be indicating trends. Beyond how far that number could go up to, it really depends on the effect size you’re talking about.

 (Suszie Tyree)

You gauge it as well from previous trials, so you might start with a small trial around 10 to 15 per group, and then based on your findings you would kind of see how big of an effect size you were getting and whether you needed to increase that in your subsequent trials, if you were developing a portfolio for a regulatory claim for example.

(Gillian Dunnglavin)

To do that we use something called a power calculator, that simply estimates the number of participants needed based on previous results. That can be previously published results, or it can be your own previous results, it’s always best if it’s your actual results from your previous work because different variations and products, different populations can lead to different effect sizes and percentage changes over time.

7. “What are the main challenges and opportunities to apply clinical data to real-world data or are they mutually exclusive?”

(Onthatile Serehete)

“They’re not mutually exclusive, no two clinical trials are the same, it really depends on the trial population that you’re trying to recruit, and why you have tailored it that way. For example, if you’re looking for people with a certain disease, some of them may already be medicated, because of the interventional product that you’re testing, you might want them not to be medicated. In a real-life situation, these are people who once they’re diagnosed, they’ll be put on treatment, so those two won’t match.

For clinical trial purposes you want unmedicated people, so we must go out of our way to find people who have been diagnosed but there’s some lapse between treatment and intervention for them. That’s just one example where there may be a mismatch, it really depends on what you’re testing and what your goals are, that would be for a trial that is really disease specific and some studies on healthy general population. Though the two may match because we’re recruiting everybody over the age of 18, various BMI’s, healthy, normal population with no specific exclusions such as particular diseases, or conditions so it really does depend.

8. “If you want to combine safety and efficacy in one clinical study, how can you better choose the study population?”

(Suszie Tyree)

I think the key is to target a general population, you don’t want to be too specific or cutting people out, some sponsors might drive for excluding people that will have some underlying condition that may look as if it’s a safety or tolerability effect, i.e., anybody with a gastrointestinal disorder or for example, if they had IBS presented with diarrhoea, then they would have a lot of AE’s with that and it may be attributed to the study, meaning some people do push to exclude people like that but generally for safety and tolerability you don’t want to cherry pick your population too much because you may run into a problem later on with making a regulatory claim where the regulatory body is going to say ‘Yes but that’s not a real real-world population and therefore you can’t make a general claim about a general population if you weren’t testing it on a general population’. We would generally recommend that you that you don’t go too restrictive for a safety tolerability trial”

 (Gillian Dunnglavin)

There’s an increased focus from regulatory bodies also, the FDA is more guidance and discussion that they’re having at the moment, they want more clinical trials to include a broader range of people, because the broader range of people are going to take it in the real-world scenarios. We’re not all healthy, it’s important to keep in mind when we’re testing safety that we’re testing on the people who will take this product in the real-world situations.

The complexity of human clinical trials can not be underestimated, each trial is unique and requires a specific approach. The benefit of partnering with Atlantia Clinical Trials is that Atlantia manages all elements from protocol design, placebo manufacture, recruitment, and study execution, to sample and data analysis, statistics and report/dossier preparation to provide a service which is technically, scientifically and clinically superior.

The clinical studies cover a broad spectrum of functional food and beverage categories, such as dairy, cereal, probiotic, different protein forms, infant-specific foods, vitamins/minerals, plant or marine extracts and medical foods. If you would like to speak to our expert team, please do so at [email protected].