Tag Archives

4 Articles

The importance of providing Labels and Structure to your Healthcare Data towards AI Success

by Srihari Yamanoor

The worn cliche: Data is the new oil!

Your organization may be big. It might be small. It might be a mature business, with an approved device pipeline, or, it might be an early stage one. Your organization might have any number of adjectives describing its nature, size and characteristics. The commonality, is that, if it is a healthcare organization (or any type of organization, though we are focused on healthcare here), it has data, that can be classified and categorized in several ways, and can indicate to a point in the immediate or distant future, where it could have significant impact and ROI depending on how it is analyzed and applied in alignment with your organizational goals.

Thus, whether the cliche seems tired to you or not, there is a lot of truth to the statement that your data can be useful in several ways in the future – it could represent inherent value for investors, it could help with forensics and root cause analysis, in new designs, in optimizing performance, manufacturing, service, competitiveness and possibly in many different ways you haven’t yet imagined! This then indicates you should have a sound data strategy.

Storage is Tactical, Not Strategic

If you think, “well, our strategy is to acquire the data, store it in the cloud and back up for when we are ready for AI”, you are confusing tact with strategy. Some might argue that the act of storing data is even more banal than being tact. There might be truth to that, but let us make sure that you do have sound plans to acquire and store data with back up plans that are better than what MySpace appears to have been employing (for the Rip Van Winkles out there, I will provide a link of the recent disaster that MySpace was engaged in – essentially losing approximately a decade’s worth of its users’ music)!

So, your strategy for data needs to be a lot more than strategy.

However, do not take storage lightly! In my career, I have seen organizations with weak leadership that engaged in such stupid ways (and this is putting it kindly) of collecting and archiving data, that they lost critical and crucial, historic information relating to verification and validation!!

So, while it might seem simple, make sure your organization has well tested and trusted methods of collecting and archiving data, backed up appropriately and retrieved securely, with reasonable ease. At a minimum, you should consult with Data Scientists, if your size and budget does not permit developing a Data Science team in-house. It might be appropriate for the Quality and/or Regulatory functions within a small organization to bear the responsibility for this. For device and drug companies, regulations already require much of this, and yet you would be surprised how easy it is for organizations to miss the mark on this.

Labeling

There are several definitions for data labeling and definition of structure, as it pertains to data. I will leave this as a basic exercise for you, the reader, but I am sure, you have a commonsense understanding of what the terms allude to, and so let us use that as a yardstick to get to the thrust of this blog, rather than reducing it to pedantic and semantic exercises. Data labeling alludes to the fact that you mark data in terms of body temperature, instrument temperature, body weight, age, sex, heart rate and on and on. For anyone who has done even the simple exercise of bench top testing, it will be obvious that you already have to take a lot of care in labeling data, or the losses can be quite catastrophic.

Labeling tends to be the most expensive and time consuming exercise in preparing your data for Machine Learning, as anyone who has attended a talk by me on this topic has been clearly warned before!

Labeling should be understood in broad terms that go beyond merely the titles of columns and rows on a spreadsheet. For instance, being able to pull several spreadsheets, and readily understand which data came from acute or chronic studies, bench-top or human trials, real vs. simulated data must all be considered. Confuse these, and your machine learning might be trained with the wrong data and the results can be quite undesirable.

Data Labeling should also be very clear at every level and this cannot be stressed enough! If you have data collected by 4 thermocouples, two years from now, when someone pulls up the data for instance, they should be able to identify exactly where the data came!

In effect, you should establish a discipline of rigorous labeling for data, from the macro levels, going into the individual levels, through your SOPs, Work Instructions, Forms and make this a part of organizational culture.

Structuring Data

Data can be classified into Structured Data and Unstructured Data. Structured Data is reflected by the type of data discussed above – data obtained manually, through machines automatically, by programming, through logs, etc. (There is a note about data logs I will make in a moment). Unstructured Data on the other hand, could be data coming from sources such as online customer reviews (if you think this is not for you, just search for reviews on IUDs or Heart Burn Medications on the internet and you are in for a surprise), physician or nurse feedback, complaint information, procedure videos and other sources. Depending on the device, drug, biologic, app or the combinations thereof, your unstructured data can take many forms, and you should develop a very deep understanding of these sources, and how to collect, and structure this data as much as possible.

While, we all assume that some day, in the near or far future, depending on the various predictions, Artificial General Intelligence will allow us to completely skip data labeling (Neural Networks can do this to an extent currently), and analyze data, providing insights. However, there are limitations with current Machine Learning and Deep Learning Approaches, and you are better off providing as much structure to your data as possible.

For instance, if you are capturing procedure videos, whether on models or on humans, you might benefit from using fiducial markers for instance. Such markers are common in imaging systems already. Your device could also have identification markers on it, that show position, location etc., making human and machine understanding fast and easy. This can be very helpful when dealing with explainability and machine training. This could essentially convert the status of your data from ‘unstructured‘ to ‘semi-structured‘ or, if you can further template and refine it, ‘structured‘. Of course, you might hit a point where you get diminishing returns, or, as is the case, the machine can handle semi-structured data well enough.

Identifying the sources of your unstructured data, and striving to provide structure to your data, early, and strategically, will ensure that Data Analysis and Machine Learning Projects will be less expensive and better facilitated.

A note on Machine Logging

I was once involved in a project where the capital equipment logged critical data such as currents, voltages and other data associated with procedures. Various regulations require this, and for Failure Analysis, Adverse Event Investigation, etc. this is generally very essential. The fundamental problem with this (and others I have seen) particular data logging was the lack of foresight and poor implementation. Starting from the very labeling of the log files, to the time stamps, to the data collection frequency and various other features, it was a complete mess. The data had to be pulled from the logs into a spreadsheet program, and a number of routines had to be run on it, to make it passably useful. I used to wonder how a diligent auditor would take to the cumbersome and questionable logging, if he or she were to stumble on it!

Worse, the project leader prided himself in thinking that he had created an innovation in translating the gibberish into a-little-less-gibberish through the scripts! That sort of paucity in planning and respect for data will not help you, if you are to succeed.

The Bigger Picture

I want to make it clear. This is not just exhortation to make machine logs readily readable. And yes, your software team MUST do this. The point here is awareness, education, discipline and diligence towards data. Your organization needs to strive towards all of these aspects – and two more, which follow in the next section. It is not just leadership, but it is your entire organization that needs to focus on this for success.

Patient Privacy and Data Security

Increasingly, ranging from Government Organizations to Data Silo Owners and even other third parties, several organizations are admitting to failures in securing data, exposing data belonging to stakeholders. As you can imagine, exposing your patients’ data, erodes trust and is unforgivable, if the root cause for this is carelessness and sloppy, outdated practices.

Even when patient privacy is not involved, losing data of any other kind, such as test data, designs, manufacturing data etc. can be catastrophic to your competitiveness, and can make or break your organization.

Efforts to secure your data and protect your patients can never cease, and there can never be complacency in any of these areas!

Conclusion

The larger conclusion I want you to draw from my post here is this: you need a Data Strategy. Within that strategy lie elements such as labeling, structuring, security and privacy, none of which can be ignored. Your organizational members need to be educated of the importance of your strategy and they must have a healthy respect to help formulate, improve and maintain the strategy and discipline it will take you to succeed.

References:

  1. Tactic Vs. Strategy: https://en.wikipedia.org/wiki/Tactic_(method)
  2. Image of Tags: https://www.pexels.com/photo/several-assorted-color-tags-697059/
  3. Image of Book Stacks: https://www.pexels.com/photo/pile-of-books-on-gray-metal-rack-1853836/
  4. Image of Escalators: https://www.pexels.com/photo/building-escalator-1769356/
  5. Image of Labeled Beans: https://www.pexels.com/photo/assorted-beans-in-white-sacks-1024005/
  6. Image of Laptop: https://www.pexels.com/photo/low-angle-view-of-human-representation-of-grass-296085/
  7. Data Loss at MySpace (An NY Times Subscription may be required): https://www.nytimes.com/2019/03/19/business/myspace-user-data.html

The FDA Designates an Eye-Diagnostic System as a “Breakthrough Device”

by Srihari Yamanoor

 

Last year, in my talk at the San Jose BIOMEDevice Conference, I had postulated that one of the key factors in the rate and level of adoption of AI based devices and systems would depend on how the FDA would act on such submissions. It is good to see that one company seems have to have proceeded with taking the FDA to task. The result is positive, and good news for companies pushing AI based systems, but also carries a tale of caution, for the road ahead for everyone involved. I will lay out the case briefly in this blog, but this is not the last time you will hear me talk about the effect of regulatory agency response to such device submissions and their long-term implications.

I just reviewed news that IDx, a company that designed what is being described as “an AI based autonomous diabetic retinopathy” detection system has been given “breakthrough” status by the FDA for their system. This provides the system with an expedited review, and potentially quick approval. This is very encouraging, not only for the organization with the retinopathy system, but also for all companies vending AI systems.

Here are a couple of key thoughts I have on this:

  1. A very promising outlook: I am not aware of how much Scott Gottlieb had, in terms of a personal hand in this decision, but I am sure his leadership had a measurable impact on the decision to expedite review. He has taken a stance of modernizing some of the approval processes, especially in the area of Digital Health. The FDA’s backing for AI based systems, is a very positive thing for companies with an AI focus on healthcare.
  2. Good for competition in the healthcare industry: Once the FDA sets AI based apps and systems on the path to commercialization, there will be no excuse for organizations that do not adopt Machine Learning and Neural Networks. Of course watch for a good spike in M&A activity in this regard.
  3. The Hype Cycle is around the corner: While companies with real ML/DL products and services will be out there, helping patients, there will also be the fakes and the wannabes that we will have to contend with. And it is quite possible that the rotten apples will ruin it for everyone. This is something to watch for.
  4. Unrealistic expectations and unintended consequences: I know this is closely related to the point about the hype cycle, but I want to make a refined point here. Even the companies with good AI based tools might push the envelope on hyping up the utility and efficiency of their products. Over-promise and under-delivery lead to angry and disappointed customers and eventually will create problems for everyone in the industry.
  5. How will CMS and private insurance companies handle reimbursement? This is a big question and can be the topic (and probably will be) of several blog posts. However, I just wanted to highlight that while AI might improve efficiency and automate diagnoses and treatments, companies have to resolve how and where the reimbursement will come from.
  6. A word of caution on regulation: While it is commendable that the FDA is moving forward with initiatives on things such as AI. However, I do not always agree they are doing a great job on things. I was irked by the craze with which the orphan applications were being reviewed by the FDA. The FDA, in the past, has gotten too cozy with the industry and Congress, with terrible consequences. Those who remember the Menaflex incident (reference below) remember how quickly the public’s trust in the FDA eroded, and how the organization took years to recover from that. Therefore, there is cause for concern here, and one hopes there will be a balancing act in making sure that the approval processes remain rigorous enough.

Conclusion: The current “breakthrough” designation and fast track review show great promise, but one hopes the FDA will balance it out with an equal dose of caution.

Subscribe and Support, Please! 

Did you enjoy this post? Please subscribe for more updates, using the sidebar. Have ideas or blog posts you’d like to see here? Contact me at yamanoor at gmail dot com.

References:

  1. The Press Release on IDx: https://www.eyediagnosis.net/single-post/2018/02/05/FDA-to-expedite-review-of-IDx-DR-a-breakthrough-AI-diagnostic-system?lipi=urn%3Ali%3Apage%3Ad_flagship3_feed%3B6Bf9xSfCRQSJXa4nvXjScA%3D%3D
  2. On what happened with Menaflex: https://www.fda.gov/downloads/NewsEvents/PublicHealthFocus/UCM183642.pdf
  3. Image, Courtesy, Pexels: https://www.pexels.com/photo/people-face-child-eye-32267/

Interesting AI+Health Imaging Investment and a few thoughts on where the field might be headed

by Srihari Yamanoor

 

This morning, a LinkedIn contact of mine shared an interesting MassDevice article about “Bay Labs”, a company focused on bringing Deep Learning to Cardiovascular Health Imaging (though they define themselves in broader terms as an algorithm company). It appears that Khosla Ventures (after the same much-hated Khosla from California), an existing investor (which must mean seed funding) has backed the organization with $6mn in series A funding.

What is interesting

  1. So far, what we have seen is, the really big companies, such as nVidia and G.E. working on backing up imaging with AI. They even recently announced a collaboration. This would have been a scary scenario for start-ups trying to break into the space. Even now, one should not take too much comfort. Bay Labs apparently won in an AI company contest conducted by nVidia. It clearly shows that big players are keeping an eye on the start up space, potentially looking to scoop up promising organizations.
  2. Though Bay Labs likes to talk about themselves in somewhat general terms; they state this on their website, “In order to serve the largest number of people, we aim to amplify the benefits of deep learning by providing high-performance algorithmic capabilities to assist with healthcare challenges on a global scale.”, they are at least currently focused on Deep Learning, as it relates to Imaging for Cardiovascular Health. Don’t get me wrong, I like both ends of the spectrum displayed here. They appear to have a broad vision of where they want to end up, and yet, as is important for a startup, they have a somewhat keen sense of focus.
    1. Given that, even with all the healthcare and lifestyle influences urged on by healthcare systems, governments, non-profits and others, cardiovascular diseases remain the cause behind the most deaths in the US and around the world. Thus, even within just this field, in imaging alone, the opportunities abound.
    2. Plus, cardiovascular health also happens to be one of the most expensive, and is a low hanging-fruit for startups, expecting to break into a new field and work towards financial success. The other field is probably oncology, which however is crowded with Watson and others. Still, I expect at least a handful of startups focused on various subsets of oncological diagnosis and treatment to emerge out of the woods as time progresses.
    3. It is also one of the fields where practitioners are prone to adopt cutting edge solutions.
  3. Bubbles: Bay Labs appears to be an organization founded on some sound principles. The next few should probably follow along the same path. One has to wonder though, at what point, in the not too far future  will we see organizations touting AI-this and AI-that, without any sound basis? This should be interesting to track.
  4. Algorithm Patents: The debate on whether software and algorithms can be patented, and under what conditions was never properly settled (what, ever is?). A new wave of companies touting algorithms will now come along, and generally speaking, take each other to court, giving headaches to Luddite Justices, such as in the US Supreme Court who are still hemming and hawing about allowing basic AV equipment within the courts (ironically AI has been trained to predict how individual US Supreme Court Judges would rule, with fairly good efficiency). This whole space is something to watch closely.
  5. Assets: Besides patents, there are other things to be concerned about here. Who ultimately owns the data? The results from what the Machine Learning and Deep Learning algorithms find? Are the findings marketable? If not, who pays to get this done? How do governments, healthcare systems and private organizations, all of which form the backbone of medicine share the costs and the benefits?
  6. Mergers & Acquisitions: Once the hype cycle is ridden and who owns what is also a bit more clear, comes the M&A song and dance. This also should be interesting to watch.
  7. Interoperability: Data interoperability in heatlhcare, is already a major headache. This is currently one of my core research areas. I am deeply interested in unraveling the challenges of how data should be cleaned and labeled to be used in AI applications. On the surface, it appears that all you have to do is collect the data, just transform it for various purposes – to upload to EHRs, to use in Machine Learning and so on and then get on with it. The underlying issues are much more complicated. Who collects the data? How is it blinded? Who pays for all this? How does the learning get applied? How do you glean precision medicine uses out of all this? And, many, many more…
  8. Regulatory Dilemmas: I have a series of blog posts planned on this topic, but let’s take a quick dive here. How do you get approvals for such AI systems? How do you fix problems? Should improvements, fed by data volume and algorithm changes require regulatory updates? What happens when an algorithm produces defective results? These and many others problems will need some type of resolution, and will create new problems and opportunities of their own.

Summary: In summary, I’d say, news of companies such as Bay Labs is good. However, headwinds lie ahead for them, and everyone in industry. We are looking at huge paradigm shifts and thus, not much is predictable. New issues will unfold while ones that seem dire right now, might become trivial. We are in for the ride of our lifetimes!

References:

  1. The MassDevice Article: http://www.massdevice.com/cardiovascular-ai-imaging-dev-bay-labs-raises-6m-series/?lipi=urn%3Ali%3Apage%3Ad_flagship3_feed%3BMBJbTpHMSb6f%2B2WzYDP%2BUA%3D%3D
  2. Bay Labs: https://baylabs.io/
  3. Image, Courtesy Pexels: https://www.pexels.com/photo/low-angle-view-of-spiral-staircase-against-black-background-247676/

Slides – How Artificial Intelligence is Changing Medical Devices

by Srihari Yamanoor

The main goal of this site is to air my thoughts on how new technologies and paradigms impact the practice of medicine. In relation to this, I made a presentation, just yesterday, December 6, 2017, on the topic of Artificial Intelligence, and the growing influence of this discipline on the field of medical devices.

The slides will be available to download this through the site.

I do have to caution you. I do not present a lot of text in the slides. I have written some notes on the slides, but together, they still will not be enough to understand all of what I spoke about.

However, do not worry. I will blog about each slide, concept and use-case presented on this presentation over the next few weeks and months. Subscribe for updates and stay tuned! In addition, if you feel you have a set of burning questions and need them answered as soon as possible, do feel free to contact me and I will answer them as early as possible.

The download link: https://drive.google.com/file/d/12o1DHzJ2f_jNJi8MzZFPqhOwydVoDU52/view?usp=sharing