Discussion
Health NZ staff told to stop using ChatGPT to write clinical notes
burnte: Yeah, no privacy or security there. There are some tools explicitly designed at helping healthcare providers produce better notes faster, and a couple of them are AMAZING. I'm an AI-half-empty guy, I'm keenly aware of its shortcomings and deploy it thoughtfully, and even with my skepticism there are a couple of tools that are just plain great. I think using LLMs to create overviews and summaries is a great use of the tech.
javascriptfan69: The one my doctor was using got my obs numbers completely wrong.We had to correct them at the end of the consultation.
samglass09: Meanwhile they are pushing AI transcription and note taking solutions hard.Patients are guilted into allowing the doctors to use it. I have gotten pushback when asked to have it turned off.The messaging is that it all stays local. In reality it’s not and when I last looked it was running on Azure OpenAI in Australia.I spoke to a practice nurse a few days ago to discuss this.She said she didn’t think patients would care if they knew the data would be shipped off site. She said people’s problems are not that confidential and their heath data is probably online anyway so who cares.
SpaceNoodled: Is there nothing like HIPAA there or what?
lights0123: Many AI companies, including Azure with their OpenAI hosting, are more than willing to sign privacy agreements that allow processing sensitive medical data with their models.
samglass09: Very little protections. The entire medical records of a significant percentage of the NZ population were stolen recently and put up for sale online. Zero consequences for the medical practices who adopted the hacked software.
Ucalegon: The devil is in the details. For example, OAI does not have regional processing for AU [0] and their ZDR does not cover files[1]. Anthropic's ZDR [2] also does not cover files, so you really need to be careful, as a patient/consumer, to ensure that your health, or other sensitive data, that is being processed by SaaS frontier models is not contained in files. Which is asking a a lot of the medical provider to know how their systems work, they won't, which is why I will never opt in.[0] https://developers.openai.com/api/docs/guides/your-data#whic...[1] https://developers.openai.com/api/docs/guides/your-data#stor...[2] https://platform.claude.com/docs/en/build-with-claude/zero-d...
simmerup: Gotta break a few eggs to save 2 minutes of thinking and work
jimjimjim: The New Zealand Chief Digital Officer allowed Australian cloud providers to be used as there weren't suitable NZ data centers and this was many years ago.
