00:00:00: You are listening to the Effective Statician Podcast.

00:00:04: The weekly podcast with Alexander Schacht and Benjamin Peske designed help you reach your potential, lead great science & serve patients while having a great work-life balance!

00:00:22: In addition to our premium courses on the Effectives Staticians Academy we also have lots of free resources for you across all kinds of different topics within ZAC Academy.

00:00:38: Head over to theeffectivestatistician.com and find the academy, much more for you.

00:00:46: become an effective statistician!

00:00:49: I'm producing this podcast in association with PSI a community dedicated leading and promoting user statistics within their scandal stream by benefit of patients.

00:01:01: Join PSI today for the development of your statistical capabilities.

00:01:05: With access to the ever-growing video on demand content library, free rep is traced into all PSI webinars.

00:01:13: Head over to the PSI website and PSI.org To learn more about PSI activities, become a PSI member too soon!

00:01:30: Welcome to a new episode of the effective statistician.

00:01:33: And today we are looking into very important thing that is making good decisions based on data and everybody in clinical research knows about it, but Today I want look at from little bit different perspective.

00:01:50: for that i have fellow podcaster here.

00:01:53: Hi Anika how you doing?

00:01:59: Very good.

00:02:01: We are recording this in the middle of summer and Anika has a new baby, so maybe there will be some noise at the background but who knows?

00:02:12: Let's see we'll be charming for sure!

00:02:14: Maybe before we go into topic Anika you can introduce yourself what brought to starting your podcast yourselves.

00:02:23: Yeah, so I'm working as a IT management consultant.

00:02:52: And you get huge data sets of around three million students with three hundred background variables, and then try to determine what is actually decisive for their learning outcomes.

00:03:02: So I come from working with big data from coding and through my work in IT management, that's actually become a manager.

00:03:10: And i'm rather concerned about how do people act?

00:03:13: With data and algorithms than the algorithm itself is working properly nowadays.

00:03:21: When I started to work on companies when I found out hey!

00:03:25: The data isn't as cleaned up or available coming from academia There's still a long way to go in companies until we can really take data-based automated decisions.

00:03:37: Yeah, very good!

00:03:38: As I mentioned you work now more in the IT department and IT area And of course, IT systems generate data all the time at every minute, every second Every reaction that they do takes.

00:03:53: The best example is probably any kind of social media data.

00:03:58: but also within companies, we leverage all kind of different data.

00:04:02: Can you give a couple examples for typical datasets that you would look into?

00:04:07: Yeah I'd say this whole idea of companies use their data very much depends on the IT architecture.

00:04:13: so how they're systems are connected with each other.

00:04:17: usually We don't have company which was set up two years ago in the ideal Designed to work with data and AI, but we have companies who grew for at least the last twenty or fifty or a hundred years.

00:04:29: And maybe they have seen mergers.

00:04:31: so companies were bought... ...and all those different IT systems just grew.. ..and oftentimes are not connected in a poor way or historic way.

00:04:41: So data is NOT flowing as freely as we would like to.

00:04:46: And there's also often restrictions, so I've worked a lot in insurance and finance.

00:04:51: So there is often regulation which has to be met that not all data can be connected to each other.

00:04:57: This is the basic setup.

00:04:58: i want to say where were working?

00:05:00: Then on how data are used in companies staircase, where we have really basic analysis of looking back what actually happened and which groups in which teams on which countries trying to see the data there.

00:05:19: That's usually the easiest analysis.

00:05:21: And still I've seen when I worked as a data scientist in consulting for companies that data has not been readily available.

00:05:30: you could now just enter your account credentials and just look at the data.

00:05:35: At least you were allowed to, but I oftentimes really had to go and collect data and talk to those people who entered the data because they where there are only ones that understand how the data came into being and how it's to be read was not self-explaining nor is any metadata to explain its documentary.

00:05:53: And then i would see second step on staircase with before casting.

00:05:57: so what will happen next?

00:05:58: The very standard example is predictive maintenance, so when do I have to look at my machines again?

00:06:04: When do i have to repair it?

00:06:07: then the third step would be correlation.

00:06:09: what moves together.

00:06:10: Is there really oil prices go up and this moves together with higher energy prices in Germany for example.

00:06:17: And the fourth question I would like to ask is that really causal?

00:06:21: or is energy prices going up in Germany driven by another effect which not revealed yet or by the analysis I've been doing.

00:06:30: And i don't really see that in companies being done, except maybe for the classic A-B testing and user design where we want to say okay on the website it's a button black or blue?

00:06:41: Or is it on the left or on the right hand side.

00:06:44: but yeah as an economist I'm very much interested in this causal interpretation.

00:06:55: I'll see that someday.

00:06:59: Yeah, many of the listeners work with RCTs all day because it's a requirement for clinical research or nearly another requirement.

00:07:08: very rarely you can get something moving forward with an RCT but not always your RCT can be run and so you need to work with observational data And then still you need do something.

00:07:22: Or if you ran in randomized some thing Really completely randomized is the RCT only very much at the beginning.

00:07:30: The longer you follow up, so more all kinds of different things happen and your randomization gets distorted.

00:07:38: let's say it that way yeah?

00:07:39: And then we need to have causal inference.

00:07:42: There's attrition when long term interactions also.

00:07:46: as far I know medical studies are not really well known for big treatment groups.

00:07:50: right It's not like you get a thousand or few thousands people in there and have really heavy statistical validity.

00:07:58: Yeah, so it depends on the treatment effect that you want to see yeah?

00:08:03: There are certain cardiovascular studies that maybe have tens of thousands of patients.

00:08:08: I would say is more the exceptions having these mega-studies.

00:08:13: most studies are smaller but nevertheless can still be very costly

00:08:19: Absolutely

00:08:20: especially when you have to observe patients over years, for example.

00:08:25: When now companies have these internal data or external data whatsoever data form making better decisions?

00:08:32: What are typical challenges?

00:08:38: So what I've seen in the past years, in companies is not that algorithm was wrong.

00:08:44: Of course you have to be careful with data and quality.

00:08:47: but most difficult step for companies when actually putting those nicely run MVP pilot algorithms at DataWorks into production and really making the users, end-users work with it.

00:09:03: Convincing them they should change their old habits of how there's always none in business that now works with data and AI to create a need for those users.

00:09:16: They really want to change their way of working and they see an advantage in it.

00:09:20: And not only there's always a handful enthusiasts for technology who would like to use that, but especially convince those who've been working the past twenty years with paper and pen... ...and probably are also not trained on a quantitative analytical field….

00:09:36: …but have worked a lot with texts or people – and to convince those and not force them?

00:09:43: that will create even more resistance.

00:09:44: but really convince them and understand their point of view, they're incentive structure.

00:09:49: They have fears it needs And thats ideally done before you've created the data analytics in AI.

00:09:57: That don't waste money on changing your AI product after talk to people But its really user opinion is already built into design.

00:10:08: Thats interesting because I see similar things in the medical research space as well.

00:10:14: Just because you have a new way of looking at the data, interpreting the data and analyzing that doesn't mean people will act differently on it or that they'll even look at it or accept it let alone use it.

00:10:31: And there's sometimes this avant-garde of people that's usually always interested in the new just for because it is a new and they will try to out.

00:10:43: but vast majority would be very pessimistic about.

00:10:47: It's uncomfortable, right?

00:10:49: I have to use so many technology tools already.

00:10:51: why should i even one more?

00:10:54: yeah And what's the value for it?

00:10:56: Why Is The Old Way Of Doing Things Not At Least As Good as A New way of doing things?

00:11:03: maybe its frightening Maybe I don't understand.

00:11:05: That's a different topic, I think.

00:11:07: A couple of years ago now it is already quite a couple of year ago in the clinical research world see whole thing around causal inference as demands.

00:11:17: things like this has been introduced and its still difficult also in observational research space.

00:11:25: what you just mentioned these more educational aspects that not new to seeing people have used propensity scores or other methods for decades.

00:11:35: But in clinical research, it's a relatively new area and people struggle with it quite a lot.

00:11:42: So what are your techniques to get people to embrace these changes?

00:11:48: I would come up with the basic tools of being a consultant right... ...I would start with a stakeholder analysis like who were the people i have to consider for my data & AI product?

00:12:00: who is involved in decision-making, whom's support do I need and when the end will use it.

00:12:06: And then i would also group those stakeholders into groups.

00:12:10: Do they make decisions?

00:12:11: Do I need to ask them?

00:12:13: Do They need to use

00:12:14: that?".

00:12:19: And to be very precise, how old are they?

00:12:22: What educational background do they have?

00:12:24: which training what incentives.

00:12:27: Do They Have What Fears?

00:12:29: and then really try To address those prioritized by How?

00:12:33: I group the stakeholders beforehand that need to consult them or make decisions so on.

00:12:39: So take Them Into The Boat Early On and i know That Something.

00:12:42: People Who Work Quantitatively Really Love Their Data and their Programming.

00:12:48: I do not really like to get into those management tasks.

00:12:51: And then I can only recommend to get help of people who can really bridge those two worlds, doing management tasks and understanding how valuable an important quantitative work is.

00:13:03: Also another step further down the road when you've done all the characterizations or few stakeholder groups in the personas than tell a story with your data?

00:13:14: Really try keep it as simple In academia, for example.

00:13:20: It's very fancy to put a lot of numbers and graphs into your slides in presentation but those people you will as data scientists then talk too present the results.

00:13:31: they have little work or attention.

00:13:34: that really is much interested not how it got there.

00:13:39: try make sure get message their needs are met because if created a wonderful data analysis and no one's going to use it.

00:13:50: You will not feel very happy afterwards because you feel like your work may be useless or not appreciated in the way that actually

00:13:58: deserves.".

00:13:59: Yeah, completely agree!

00:14:00: And I love your point about the personas is there are lots of incentives for them as well as their pain points.

00:14:08: so we can talk kind of titles and gender-age whatever.

00:14:12: but these two things Variables have a huge influence.

00:14:18: And if you don't know what these are, it becomes really difficult because you need to understand how can convince them use it and that is by addressing an incentive.

00:14:32: maybe say incentivize doing something cheaper or faster.

00:14:37: That could be something.

00:14:38: or maybe they're incentivized by being innovative.

00:14:42: Could another thing?

00:14:43: Or maybe they have a pain point that certain things take very long.

00:14:48: And then some kind of dashboard helps to get through the data much faster than hundreds of PDF tables?

00:14:56: Yeah, I've worked with users who are not really incentivized that they get paid more or something when they work faster and better and more innovative but just feel there's such feeling of responsibility.

00:15:09: They really want to cope with their work and do their work, but they are at the same time so overwhelmed because there's so much work waiting for them.

00:15:17: And then after feeling I need to push on... ...and don't have the time to learn a new tool that will just cost me time!

00:15:24: It is really about showing you how it works.

00:15:28: They'll save more productive times and work better when they've gone through training for this new tool.

00:15:34: They feel pressure from in a better way afterwards.

00:15:39: No, we talked about the word persona and actually didn't explain what that is.

00:15:45: so for anyone who's listening doesn't directly have concept of mind as to how would you describe or explains our persona?

00:15:55: Yeah I'd really work with pictures or at least comic picture of person For every relevant stakeholder group.

00:16:04: i would create a persona Even among the users, this could be two personas when I have to describe distinct groups among my users.

00:16:13: And that's why talked about age and gender and so on.

00:16:17: it will really have a concrete person in mind to have it always In my mind When i design The algorithm or the output of the algorithm Or the presentation Of My analysis all the dashboard?

00:16:33: those pictures of the people in that certain group and then also design a picture for them, maybe with the close of their persona or target group.

00:16:45: And put around other characteristics as incentives to make it clearer what they actually do all day and which data maybe do they need, in which they don't need.

00:17:02: And there it's also very helpful to talk with those people who look over their shoulders.

00:17:07: you can do that on place or office but you could also virtually online share the desktop show what they actually do In which systems?

00:17:20: Yes these personas are basically resemble a typical segment subgroup of some people said and having that in mind makes it easier to communicate with them.

00:17:33: When I do my communication as well, you know?

00:17:36: Sometimes even a specific person in mine says they want to talk too like Peter or Claire or whoever.

00:17:44: yeah And then i talked to said person because I know the person stands for a specific segment of listeners.

00:17:52: If we ask more quantitative people One, convince others and not just stay in the purely technical area.

00:18:02: What kind of typical skills would you see what we need to be successful?

00:18:08: That's a big question!

00:18:09: A few podcast episodes recently on The Skills for the Future And I've also been just on a career talk last week where i had young data scientists analytical people telling me there are LLMs.

00:18:22: now They actually do the work I like, which is fun for me.

00:18:26: What job am i going to take now that they've taken away my dream job?

00:18:30: what Am I gonna Do Now?

00:18:32: and also of course cannot look into The future I can just tell you, try to find out what other passions you have and where your strong in.

00:18:42: And of course we will always need people to check the code but not as many as we had in past few decades... ...and trying to train a bit more on our people skills.

00:18:53: exactly that's talked about.

00:18:55: On one hand side preparing for data and AI projects To have the right people onboard The right setup for your analysis.. ..and then on another Also, to be able understand the business or underlying questions you're analyzing.

00:19:13: And then also take roles like working with other people becoming a team leader.

00:19:18: those are things.

00:19:19: being in physical world and working for now will not taken away by AI but we certainly have react.

00:19:29: this change on our labor market We shouldn't ignore it.

00:19:32: I completely agree.

00:19:34: Yeah, you mentioned a couple of important points.

00:19:37: Understanding the business how it changes and that doesn't not only mean how theoretically your company should work but how it actually works.

00:19:47: who are the distribution makers?

00:19:48: How our decisions make?

00:19:50: what are pain points within your company?

00:19:53: where is innovation going?

00:19:56: ?How can you drive innovation forward?

00:19:58: The listening part Is also really important.

00:20:01: we talked about the personas.

00:20:04: How do you get to these personas?

00:20:05: by listening to people?

00:20:07: A hugely underrated part.

00:20:09: Because without that, go nowhere!

00:20:12: You can be the best presenter.

00:20:14: if your not a good listener first... Your presentation will fall flat because actually don't know who are we going talk about.

00:20:22: Absolutely yeah.

00:20:22: and there is what needs.

00:20:24: again What does your counterpart need And then configure what you have for certain audience?

00:20:33: I once saw this meme about LLMs and programmers, it was something like people don't know what they actually need.

00:20:42: We are safe!

00:20:44: And there's something true about that.

00:20:46: as a technical expert you KNOW WHAT THE NEEDS ARE but also what other opportunities or possibilities bring these together is very unique skill.

00:20:58: And to get this out of the people, right?

00:21:00: You could also send around an AI generated questionnaire.

00:21:04: But we know the respondents rate is not so very high with online questionnaires.

00:21:10: It's higher when you do it face to face, person-to-person and then maybe there are new questions coming to mind.

00:21:16: You're gonna interfere their go deeper into a topic which couldn't anticipate or people only feel comfortable sharing what they true beliefs values needs comfortable office, maybe over a coffee and you talk in more informal way.

00:21:34: And then only he will get to the ground of their true feelings on what's important for them how they make decisions not from having formal business meeting or filling out those questions?

00:21:45: They completely agree because one has lot trust in it but other doesn't.

00:21:52: I think people are far away from trusting an AI counterpart which is your point

00:21:59: And that's good.

00:22:01: We should keep checking on what they are doing and always take it with a grain of salt because if we would completely trust them, then I think you will lose our competitive advantage.

00:22:10: What else can reshumans bring to the table?

00:22:14: Thanks so much for this discussion Anika!

00:22:17: I really enjoyed it.

00:22:19: Thank You For The Invitation.

00:22:21: I need to say, i have a huge respect for you recording the podcast episode.

00:22:25: Well that's at same time being here young mother and taking care of your little one.

00:22:30: Shout out to you all for this one!

00:22:32: And i can recommend people to listen to Anika's podcast as well.

00:22:38: just remind me what was... it has like pretty nice name.

00:22:43: Datenaffäre!

00:22:44: Have look into it.

00:22:46: we'll put links in show notes.

00:22:48: Thanks Anika.

00:22:55: This show was created in association with FiatSign.

00:22:58: Thanks to Rain, who helped us showing the background and CQ for listening.

00:23:03: Reach your potential, we create solutions in such a case!

00:23:07: Just be an effective statistician.