00:00:00: You are listening to the Effective Statician podcast.

00:00:04: The weekly podcast with Alexander Schacht and Benjamin Piske designed to help you reach a potential, lead great science and serve patients while having a great work-life balance!

00:00:22: In addition our premium courses on the Effectives Staticians Academy.

00:00:28: we also have lots of free resources for you across all different topics within Z-Academy.

00:00:37: Head over to theeffectivestatistician.com and find The Academy, And much more For You To Become An Effective Statistician.

00:00:48: I'm producing this podcast in association with PSI A community dedicated to leading and promoting user statistics within the healthcare industry With benefit of patients.

00:01:00: Join PSI today to further develop your statistical capabilities with access to the ever-growing Video on Demand content library, free registration for all PSI webinars and much more.

00:01:13: Head over through the PSI website at psiweb.org To learn more about PSI activities.

00:01:19: you can become a PSI member too quickly.

00:01:30: Welcome to another episode of The Effective Statistician and today I'm really, really looking forward speaking with Anna.

00:01:39: And before we dive into the topic, Anna maybe you can introduce yourself towards your audience?

00:01:46: Yeah thanks for having me happy to be here!

00:01:50: So i am Anna, I am physician by training and started my career as a doctor.

00:01:55: very fast moved to pharma industry where I've always wanted to work, so it was my dream job.

00:02:01: And i worked across several therapeutic areas as a clinical program lead.

00:02:06: So I planned the trials and ensured that The trial results are interpreted correctly Also communicated with regulators and health authorities.

00:02:26: And I want to make a disclaimer for this conversation that everything I'm saying is reflecting solely my personal views and not the view of the eye.

00:02:37: Very, very good!

00:02:38: By the way i've worked for Burringer England as well.

00:02:41: it was actually in my first company that had joined two thousand two after my academic time at my PhD.

00:02:51: good close relationships, especially as C-headquarter is just about an hour away from me.

00:02:57: So that's very nice!

00:02:59: Today we want to talk a little bit about the expectations that physicians like you have on statisticians and other quantitative scientists set.

00:03:10: work together with you as a physician at our clinical program lead.

00:03:15: let's talk a Typical challenges that you've seen in your career, In terms of statisticians.

00:03:25: I have always been blessed working with beautiful colleagues and all the companies i worked on And I never could complain about quality.

00:03:33: But what are my main things to expect?

00:03:38: Which sometimes was a bit of luck.

00:03:40: So...I would expect real partnership.

00:03:43: There is sometimes a bias in clinical trials that the clinical program lead is responsible for everything and making their final decisions.

00:03:52: This is true to some extent, but I'm always very much enjoying when statisticians bring up proposals where they point on my mistakes or white spaces which i don't see.

00:04:06: And we had fantastic collaborations with colleagues who actually brought us new trial designs.

00:04:12: It was much better than I initially thought of.

00:04:16: So, kudos to these people!

00:04:18: And another thing that i would have in mind... The decisions are usually lacking the medical background which is very expected and normal but for me it's very important to have their end goal in mind.

00:04:33: so why we're doing clinical trial?

00:04:34: Not to take the boxes not fill the database But you do bring your drug to a patient to bring a valuable drug to the patient.

00:04:45: So I always try to speak more about why are we working, who is waiting on the other end and why all that data really matters?

00:04:55: I cannot accept when some data is lost just because of some typos or incorrectness in completeness of CRF Because this is other people's work.

00:05:04: it's another peoples life And for me its important that all these statisticians understand this at least to some extent when they're working.

00:05:12: You brought up a couple of really, really important points so the first is to have this end in mind.

00:05:18: what do we want achieve with this study and be very clear about it?

00:05:22: Is that studies set...to make an internal decision in terms of said we can move from phase two through phase three or something that will need for filing such as the pivotal?

00:05:35: Is it the studies that we need for HKA purposes?

00:05:39: Or do we need this study to take multiple boxes.

00:05:44: And getting all of these very, very clear is I think one of some main points because then you know how good looks like and without that It becomes really difficult To Do All The Other Things That You mentioned.

00:06:00: So how do you work together with the statisticians so that there's a very, very good alignment on understanding what good looks like?

00:06:09: First of all I try to set up our relationship at night level from meeting one.

00:06:15: And i always start from please challenge me because especially when we walk across different geographic regions Especially if your think about colleagues from China where it is not okay in their culture someone who is considered a team lead, you need some special attention to that.

00:06:33: and I adore when people challenge me because something great always happens after these conversations.

00:06:39: Also at the very first meetings i try to set up with a frame so i tried to explain what all these terms mean, where they suffer from and then I set up the context for this trial.

00:06:56: so it's very rarely isolated as you said.

00:06:59: It leads to something and it has a history, the background.

00:07:03: So I try to always explain where this trial was born from why is it conducted?

00:07:08: And what we aim to achieve?

00:07:10: What may be the spin-offs of these trials in

00:07:12: future?".

00:07:13: This usually helps set up the level of scrutiny when you look at data cleaning... It also helps meaningful collaborations.

00:07:24: so people are more willing to work if they feel like they contribute into something great.

00:07:29: Yeah, absolutely.

00:07:30: I was always feeling like if i know why I'm doing things... ...I am much more willing to do them and also it makes it possible for myself to prioritize things.

00:07:42: So ask me Why we have that part of the study design?

00:07:46: Why does this need an exclusion criteria or a questionnaire?

00:07:52: And think about wouldn't be better than having different designs in anything shape or form?

00:08:00: Do we need an active comparator, do we need a external comparator...do we need anything so that we can run in indirect comparison.

00:08:09: Like maybe if you have a typical questionnaire here and the study but all of these external publications are run with different questionnaires.

00:08:18: it would be much harder to make direct comparisons this way.

00:08:22: How long does the study needs to take?

00:08:24: There's many things to think about And having that psychological safety, so you can talk about these and ask about there's a better way.

00:08:35: That is really helpful!

00:08:37: I would also add to this maybe missing data.

00:08:40: So working with the missing data especially in oncology You usually have different approaches.

00:08:47: Having them in mind helps the statisticians together with CPL To decide what their most optimal ways

00:08:55: are.

00:08:55: Yeah, so especially in oncology also switching data.

00:08:58: You need to understand how that impacts the safety analysis?

00:09:04: What is right as demand explaining these estimates and a good way?

00:09:08: one other thing you mentioned is proactive behavior from the statistician.

00:09:15: what do you think statisticians need To be more proactive?

00:09:20: I Think it should be the clear responsibility share.

00:09:24: So when I was talking to my colleagues in biostatistics, I sometimes hear sentiment that much more is expected in terms of proactiveness.

00:09:35: That's the zone control for this person and i'm very compassionate because of course final decision on end point lies at clinical program lead And I feel like speaking about psychological safety, if a statistician understands that it will be not his or her final decision.

00:09:56: They'll feel much safer to propose different solutions.

00:10:01: so no feeling this pressure to apply clinical filter which people don't have the health are not feeling this regulatory pressure And knowing that they always can consult with other colleagues, and we sometimes have shared responsibility.

00:10:19: Or to perceive it as contribution not a final step I think helps proactiveness a lot.

00:10:26: What also help is when the relationship are established in ways you're not only speaking that people must be proactive but really create this trust.

00:10:38: So the episode happens and you accept this, your welcome is feedback.

00:10:43: Another time another time every time it becomes easier and easier And sometimes my colleagues just came up with something I was not thinking about at all For example additional evaluation of probability or success of different scenarios or additional schema of endpoints analysis, like the sequence and alpha spreading.

00:11:06: So it's something not usually in my head as a clinical program lead but these were very valuable discussions And we involved cross-functional colleagues.

00:11:15: for example what to do you assess first PFS or PROS?

00:11:19: That is actually good question right!

00:11:21: It doesn't have sometimes only one correct answer.

00:11:24: Yeah, I think that's a very good final sentence.

00:11:27: It says usually not black and white.

00:11:30: there are lots of different things you can do And if you raise your proposal as an idea even though maybe it is really well thought through but you're pretty sure about the right way to go.

00:11:45: You still have an idea for discussion.

00:11:49: That makes it much easier for everybody else and then it becomes much more of a team decision.

00:11:57: Because many people will have different views on that, yeah?

00:12:01: How feasible is from an operational point-of-view?

00:12:04: what will the regulators say about this?

00:12:07: would be different in Suez or Europe?

00:12:10: how we'll be viewed from a market access point view or positive meta for such study?

00:12:16: What are the physicians' patient's points?

00:12:19: So there's always lots of different considerations so that you can overall come up with a good solution.

00:12:27: And from my personal perspective, I would even recommend to bring this up earlier rather than later.

00:12:33: You don't need to bring out the polished and very well thought-through decision.

00:12:38: Just bring it as early as your initial idea.

00:12:41: Let us challenge it let test its viability co-create, we will make a common decision which everyone would be happy with and you'll not have the frustration of spending plenty time on something that is denied.

00:12:57: I know from many trainings this is major topic so people participate in meetings it's fast paced discussion and the statistician has an idea or... This is different point to do but is hesitant to share it just because he may not have completely thought this through.

00:13:23: And then the discussion progresses and afterwards, so statistician thinks like maybe I should've said something?

00:13:32: But that's too late!

00:13:34: So i encourage people speak up even if you're not a hundred percent sure and that it's much easier in team cultures, as you have just discussed.

00:13:46: Where there is a lot of trust where there are open debate everybody can contribute Everybody is recognized and respected.

00:13:55: In such culture It is much easier to come up with let say half-baked solution That we put out here as the question.

00:14:04: So if you think about The typical skills of set decisions These are usually the quantitative skills of well-trained, methodological people.

00:14:15: And we talked about a couple or let's say attitudes that they need to have and things you do differently in terms for example being more proactive... ...more willing to share ideas not completely solved

00:14:32: through..

00:14:33: ..and think about all different things having the end in mind.

00:14:39: What skills do you think statisticians need to accomplish these and be really effective in such team environments?

00:14:49: I would not speak a lot about specific methodology's kills.

00:14:52: So, In my experience... And personally are not very well-familiar with all the theories that statistician know but what i personally challenge constantly can we have analysis earlier Like, because time is money and time has reached to the patients.

00:15:12: And honestly if you're working in a late-stage environment—late stage development —you will constantly be asked can we incorporate interim analysis?

00:15:23: To get results faster...to get approval faster…and so you'll be asked a lot of details.

00:15:30: what would be Why is this and why not that?

00:15:37: What's your previous experience with Regulator, so the skill to summarize its experiences and explain it in a lay language for people who are not deep into start topics.

00:15:48: This is very valuable skill!

00:15:49: If we're speaking about early stage development – I'm mostly talking on oncology first-in-human or phase two non-oncology – here flexibility is gold.

00:16:01: Sometimes I know that people like mathematical environment tend to be very detail-oriented and very data driven.

00:16:11: But the questions, what if happens all of a sudden?

00:16:16: And i would ask for example is this model best to describe whats going on ?

00:16:23: Is this approach to dose escalation is optimal one?

00:16:26: can we do faster?

00:16:28: Can We Do This More Precision?

00:16:30: If we discuss Bayesian statistics, I would challenge what you put into the prior.

00:16:35: And... ...I want explanation because i won't understand What am doing and need to present it?

00:16:40: I need to defend It!

00:16:42: And I need explanation in lay language.. ..and I NEED the willingness & readiness To explore all these multiple scenarios Or explain why they are not worth exploring.

00:16:53: But an early development will never be a case when You have set up something and forgot about that for two years.

00:16:59: It will be a constant challenge, constant protocol amendments and this is pretty normal.

00:17:04: And very valuable.

00:17:05: when statistician could support you on this journey

00:17:08: I hear the first one saying explain things in Lehmann's turn.

00:17:15: From my point of view This Is One Of The Most Important Things Statisticians Need To Be Doing Whenever I interview statisticians or have interviewed statisticians for new jobs in the last twenty years.

00:17:29: One question was, imagine I'm a physician and have no clue about statistics?

00:17:35: Please explain to me what a p-value is!

00:17:39: And then getting something that's understandable for nonstatistician... ...and still mostly correct.

00:17:47: It doesn't need to be one hundred percent correct but mostly correct is super valuable And that usually defined for me, is there a statistician?

00:17:57: That really understood things and was able to explain them.

00:18:03: This p-value question comes up again over my career also being asked as a statisticians or different people from marketing and market access.

00:18:14: Our favorite questions are what hazard ratio is.

00:18:17: Could you explain the hazard ratio?

00:18:22: This is what every medical and marketing person would ask you.

00:18:26: Yeah, yeah very good even understanding what a hazard is.

00:18:30: it's already big thing in terms of hazard ratio.

00:18:33: instead you have non-properational hazard ratios things like this.

00:18:38: oh my god.

00:18:41: so all the theory is worthless if we can't communicate that clearly.

00:18:46: My experience is If You Can Explain These Things to Non-Statisticians To Your Colleagues This is highly appreciated by everybody.

00:18:57: And it builds a lot of trust, builds trust in your competence and builds trust into all the relationship also because you can show that you actually care for other non-statisticians.

00:19:11: This are those soft skills which make you outstanding?

00:19:16: Because hard skills many people have.

00:19:18: but this trust was built exactly when I first asked simple question and received a simple answer, even slightly incorrect in very detailed manner.

00:19:28: But yeah this is something what makes people want to work with you?

00:19:31: What makes people invite you?

00:19:33: the meetings of the wider team so that they can present themselves more and more?

00:19:40: For me it went also towards the point where as I was working with I needed to present findings.

00:19:52: he would take me along and then we would present the data together.

00:19:56: And whenever there was any tricky questions like why didn't you do that way of different ways of alpha-spending or did we use set stopping rule?

00:20:07: You can directly explain to it, especially when they are coming from another company or maybe they have rather competitor publication and they've done it in a different way.

00:20:21: You can explain why you did your way, not the other way on the spot.

00:20:27: this is incredibly helpful And your physician usually really thankful that he or she doesn't need to answer these technical questions.

00:20:37: So people's skills Soft skills are very important In terms of set.

00:20:42: which soft skill do you see being most important?

00:20:46: So a set ability to explain, the ability to ask questions and understand what's exactly needed because the layperson sometimes cannot explain you in statistical terms.

00:21:01: What is needed?

00:21:01: But for example I would come say that I need to show superiority on several subgroups And i'd be so much happier if Propose me something and show some schemes, explain why this isn't.

00:21:15: that would work.

00:21:17: And maybe say a few subgroups I don't need is too much for the trial?

00:21:21: And propose another way to approach it... Maybe some exploratory endpoints or sensitivity analysis.

00:21:28: So skill to get exactly what's behind question and sometimes answer questions not asked.

00:21:37: This is very valuable Or at least making me to come as a clear question for you.

00:21:43: Maybe it will be not the first question!

00:21:45: I love that point, i've seen said again and again in discussions.

00:21:51: yeah there's an initial question but this is NOT THE ACTUAL QUESTION.

00:21:55: This just maybe things that certain person used getting or something like this And understanding why behind makes thing so much easier.

00:22:06: What would u do with it?

00:22:08: Who'll use?

00:22:09: Who's the dissolution maker?

00:22:11: What are the

00:22:11: rules?".

00:22:12: Things like that.

00:22:13: I once had a student in my leadership program and he was preparing safety results, and Excel spreadsheets to go back to their physician.

00:22:24: It also some early phase studies And through my programme He got encouraged To speak with his safety physician.

00:22:31: Ask him what you're doing.

00:22:34: And then the safety physicians showed him what he's doing with it and how is exploring these excellent spreadsheets, comparing data.

00:22:43: The decision said that these excellent spreadsheet are really a sub-optimal tool to do all of this... ...and programmed our Shiny application to display the data which reduced working time for those physicians on an ongoing basis from one day until ten minutes.

00:23:02: Fantastic!

00:23:03: and then he recommended that to all the other physicians doing similar tasks on similar studies.

00:23:09: And it rolled out across the company, so just because someone dared say what do you need this for?

00:23:17: What are the tasks behind

00:23:18: them?".

00:23:19: So they can come up with a tool that fits much better than

00:23:24: we actually love visual tools!

00:23:26: I was so happy when my beautiful colleagues introduced plenty of my exploratory analysis so that I stop pinging them for each and every question.

00:23:40: And it saves all of us so much time, and i could do so much exploration on myself because it's so valuable!

00:23:47: Yeah...I think every study at the end needs some interactive data visualization because exactly for exploring subgroups, exploring additional analyses ...and that speeds up time Like nothing, you know?

00:24:03: And also it saves a lot of additional programming time and all these other things.

00:24:09: In this topic I usually come up with some hesitance... Now i meet some hesitances about this exploratory analysis is not validated.

00:24:18: You cannot publish it or bring to the regulator but that's just not the point.

00:24:23: So all those exploratory analyses are purely for internal understanding.

00:24:28: What went wrong?

00:24:29: what were the signals?

00:24:31: Where could we look next?

00:24:33: What can be utilized for our real-world evidence

00:24:36: studies?".

00:24:37: And there are very different goals and a very different understanding of all this regulatory stuff, the pure creativity and exploration stuff.

00:24:46: So I hope that is an understanding on it with less hesitance to provide these unregulated tools.

00:24:56: Yeah, and very often you have let's say twenty different endpoints.

00:25:00: You have ten different subgroups... ...and maybe a couple of different populations that we'll actually look into.

00:25:08: That creates huge piles of tables.

00:25:12: Understanding these tables is not set straightforward.

00:25:16: Even putting all those numbers in the table into good tools will help you a lot And also understanding how data has changed.

00:25:25: Like you have mentioned oncology, different data cuts.

00:25:29: How did the data change across to different data cut?

00:25:33: Are there any new things like that?

00:25:36: Safety is always a great tool for data exploration... ...to understand what's going on so as it can get a sense of what was happening in the data again and again.

00:25:48: The more you get beyond the regulatory things, it's really important to explore your data and understand your data.

00:25:56: Absolutely!

00:25:57: Talking about this gives us that we're also working in oncology... ...and are now in twenty-twenty six years old.

00:26:04: I think there is one topic which can't be avoided And that is EUJCA and how that impacts all our work And also the statistics work especially.

00:26:16: So in your environment, how has EUJCA process impacted you?

00:26:23: Yeah I've actually experienced that already and i would say mostly with our HR manager rather than statisticians but it requires a new way of working.

00:26:38: Maybe one of the main problems is that now we need to acknowledge all these different standards of care in different countries.

00:26:46: And my yet limited experience with UGCA, I would say that Germany sometimes have strange standard-of-care in their official documents.

00:26:56: It doesn't match with the guidelines.

00:27:00: so i'm not sure what's logic behind.

00:27:03: but We need respect and acknowledge while trial planning.

00:27:07: And it creates additional difficulties when you plan the global trial across different regions with very different standards of care.

00:27:15: and actually, You cannot just put everything into your control arm.

00:27:19: Here is quite a cross-functional effort to decide what exactly would be put in the control arm What would be statistical assumptions for the control so that we don't underestimate the control performance at the end.

00:27:33: and also we would need to plan additional comparisons, indirect comparisons literature research I don't know some meta-analysis or real world studies to satisfy the regulator at the end.

00:27:47: And for me this is a major challenge.

00:27:50: The other thing that was involved in preparing for the dossier submission much in advance.

00:27:56: so actually you have to do this at risk.

00:27:59: You have to invest effort and time when you don't know yet if you submit your drug for approval at all.

00:28:07: This is what I don't like about this process, because it blocks a lot of colleagues for some time.

00:28:13: but that's the reality we'll work with.

00:28:16: maybe will change sometime in future.

00:28:19: Yeah completely agree!

00:28:26: understanding what is the guideline, but also understanding what are typical standards of care in different markets.

00:28:35: For example, author-readable evidence can help you quite a lot.

00:28:39: You could run retrospective analysis on German claims data to understand whether treatment has been used and if there may be some treatment set.

00:28:48: The GBA thinks about it as important or not that important?

00:28:52: Then maybe we can exclude these things.

00:28:55: What are the different doses that are managed there?

00:28:58: How long is it treated, what are the patient populations?

00:29:01: as another really interesting point.

00:29:04: Making sure you have outcomes said or accepted across all these different countries.

00:29:10: so in a sense also what's called PECO statement has set clear and they're both targeted as well as systematic literature reviews which will help to and from your trial design but also help with indirect compositions, network metalluryses.

00:29:29: all these things you need to basically have prepared.

00:29:32: Last year I was in an APF meeting here in Germany that was all about set new part And one of the key messages was You can't start early enough With all the work because it's a huge amount of work.

00:29:53: The takeaway is also the statisticians that work on these HDA things needs to work hand-in-hand with people who work in studies and clinical submissions.

00:30:05: And I understand it's a very different skill set for the statistician,

00:30:13: Yeah, it's a very different skill set.

00:30:14: You usually also come from a very-very different sort process and of course that would be optimal to have someone who knows everything but I haven't yet come across such unicorn statisticians in both worlds inside art.

00:30:30: Any final point Anna?

00:30:32: That you'd like keep the listeners?

00:30:42: Ability to have some ownership and ability to communicate with the other colleagues at the eye level is something that makes outstanding statistical leaders versus those who stay as contributors.

00:30:56: So if you want to develop into leadership, soft skills communication translation from start language to lay-language person understanding why it's behind questions This is all so crucial.

00:31:12: And I cannot judge on the hard skills which are needed to become a beautiful expert, but definitely people who master in both the heart and soft skills such as Gem in our organization sent it's great pleasure working with them!

00:31:28: Thanks so much Anna.

00:31:29: i can only reiterate what you have said.

00:31:32: thanks for having me.

00:31:34: Thank You my pleasure.

00:31:41: This show was created in association

00:31:44: with PSI.

00:31:45: Thanks to Reign and to our team at VVS, we are position the background and thank you for listening!

00:31:51: Reach your potential, leap right forward and serve patients – just be an effective statistician.