A cultural shift enables the full ML lifecycle

Photo by Jonatan Pie on Unsplash

Productizing ML Models

For each model you’re deploying, your entire org should treat each like a product. Decide what you are *jointly* working toward your engineering, product, and revenue counterparts. To do this, you’ll have to write out goals and requirements as well as the business or customer-facing outcomes you intend to create. Instead of generic goals, like better accuracy and precision or arbitrarily faster deployment, tie your goal to what makes your features, model and code into a product.

  • What product promises have you made? Typical product promises you might make, like these from Chase, would be “We monitor your account 24/7 using sophisticated real time fraud monitoring and can text, email or call you if there’s any unusual activity on your account.” and “With our Zero Liability Protection you won’t be held responsible for unauthorized charges made with your card or account information.”
  • How fast is fast for your application? Typically, fast looks like predicting fraud as a purchase is being made and denying it before the charge is is approved and contacting the customer with enough time for them to approve it before they walk away from the register
  • How accurate do the models need to be? In this case a bad impact of a false positive might be that I can’t buy necessary medications, food, or gas. But a false negative means that the bank could lose money by approving a transaction that is later marked as fraud by the customer or by the bank after using a series of transactions to flag potential fraud.
  • How often do the behavioral patterns you are trying to measure change?
  • Credit card fraud patterns change seasonally, annually based on promotions and cost of living increases, as well as based on events. Events are much less predictable like the pandemic, predicted hurricanes, outcomes of an election.
  • The evolution of technology can also change the likelihood of fraud. When the chip and pin was introduced the theory was that in most cases fraud wasn’t possible when using a chip, but this is no longer the case.
  • What groups benefit from your model and what groups are disadvantaged? Depending on the choices I make in feature engineering for these predictions, different groups may be disadvantaged more than others. We would need to measure disparate impact to know for sure.

--

--

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store
Charna Parkey, Ph.D.

Charna Parkey, Ph.D.

VP of Product @kaskadainc. Startup leader. Speaker. EE-PhD in DSP. Formerly #6 @textio.