Integrated MLOps: Accelerating AI-powered Finance with Domino Data Lab and VMware PDF Free Download
1 / 38
/38
100%
1
Paul Nothard
Principal Solutions Architect, Financial Services,
VMware
Yuval Zukerman
Sr. Director, Head of Content, Domino Data Lab
#vmwareexplore #INDB1953BCN
INDB1953BCN
Accelerating AI-powered Finance with Domino
Data Lab and VMware
Integrated MLOps
Confidential │©VMware, Inc. 3
•This presentation may contain product features or functionality that are currently under
development.
•This overview of new technology represents no commitment from VMware to deliver
these features in any generally available product.
•Features are subject to change, and must not be included in contracts, purchase
orders, or sales agreements of any kind.
•Technical feasibility and market demand will affect final delivery.
•Pricing and packaging for any new features/functionality/technology discussed or
presented, have not been determined.
Disclaimer
Confidential │©VMware, Inc. 4
Agenda Who are we?
AI is great!
Headwinds: Why did I even try AI?
MLOps and VMware to the Rescue!
Taking the initiative
Closing
Confidential │©VMware, Inc.
ABOUT ME
5
Meet Your Speakers
A quick introduction
Paul has over 25 years broad IT
experience, a majority of it
working for and with the financial
services sector.
He is a CS-kid who did AI back in
the early 90s before it was trendy.
Having been with VMware for
over sixteen years, Paul brings
pragmatic and innovative VMware
based solutions to our Industry
customers.
Paul Nothard, VCDX
Solutions Architect,
Global FSI Solutions
Confidential │©VMware, Inc.
ABOUT ME
6
Meet Your Speakers
A quick introduction
Paul's got IT skills that pay the bills! With over 25 years
of experience, he's like a tech wizard mixed with a
rockstar.
Working at VMware for over sixteen years, Paul’s seen
more game-changing technologies than a kid at an
arcade. People come to him for advice because he
makes tech practical and relevant to the business, like a
tech therapist.
Paul's latest gig is advising financial services customers
on hybrid cloud strategies. It's like he's the cloud
whisperer! By combining infrastructure and development
team maturity models, he'll have your business soaring
through the cloud like a superhero, cape and all.
Paul Nothard, VCDX
Solutions Architect,
Global FSI Solutions
Confidential │©VMware, Inc.
ABOUT ME
7
Meet Your Speakers
A quick introduction
A technologist by trade, Yuval
transforms the latest innovations
in AI to human-friendly content.
Prior to the current role, spent
time at VMware, MathWorks,
Brightcove, Four Seasons Hotels
and Isobar. 5 industries in 22
years.
Yuval holds a Master’s degree
from Harvard University and a
Bachelor’s Degree from UCLA.
Yuval Zukerman
Sr. Director
Head of Content
Confidential │©VMware, Inc.
ABOUT ME
8
Meet Your Speakers
A quick introduction
In the city lights, where silicon gleams,
Yuval walked through bytes and dreams.
A technologist, sharp and neat,
Turned AI tales to rhythms sweet.
From VMware’s cloud to MathWorks’ code,
Down Brightcove lanes, his stories flowed.
Four Seasons saw his techy finesse,
While Isobar caught his digital impress.
Five industries, twenty-two years' haze,
Through tech’s vast, shifting maze.
From Harvard's halls to UCLA's shore,
Gathered wisdom, forever more.
The West Coast waves, the East Coast charm,
In both, Yuval found no harm.
With a Master’s cap and a Bachelor’s tale,
He journeyed on, without fail.
Yuval Zukerman
Sr. Director
Head of Content
9
AI is really Great !
Confidential │©VMware, Inc. 10
AI is having an impact. Right now. Today.
37% seeing AI-driven
cost reduction2
AI Adoption by business function1
Life Sciences
Data Processing
Financial Services
1
2
Top Industries
investing in AI3
3
AI Adoption Impact
79% deployed for three
or more types of AI
applications at
scale2
94% See AI as critical to
success2
$249B $48B
$98B $25B
AI Investment 2013-20223
seeing AI-driven
revenue increase1
63%
Confidential │©VMware, Inc. 11
Finance is beyond just dipping its toes
24% of financial institutions
use AI.447% adopted robotic
process automation3
42% adopted natural language
document processing3
33% use virtual agents3
Confidential │©VMware, Inc. 12
And AI makes you money
What an Accenture study tells us5
10% 22% 31%*
2018 2021 2024
"AI-influenced”
revenue
Sales of existing products and services
made possible through better AI-driven
insights on customers, supply chain, channels
Sales of new products and
services made possible by
human + AI
Sales include some cannibalization and net new revenues. Excludes higher efficiencies in
production operations thanks to AI.
Higher prices through dynamic
pricing ML algorithms.
13
Why did I even try AI?
Confidential │©VMware, Inc. 14
AI comes with Risk
To name a few…
Data Privacy and
Security
Model Risk Operational Risk
and Overreliance
Ethics and Bias
Regulatory and
Compliance
Retraining and
Displacement
Financial Risk Cybersecurity
Risk
Reputational
Risk
Confidential │©VMware, Inc. 15
Is your data ready for AI? Are you ready?
Siloed dataData Quality Scalability Too much / too little
Data Controls Data Scarcity Data Privacy
Confidential │©VMware, Inc. 16
Laws and regulations are catching up with AI
Do you need even more compliance?
123 AI-related bills
worldwide3.
127 Countries with AI-
related bills3.
88 AI-related federal bills
proposed in 20223.
57 AI-related state bills
enacted into laws3.
Confidential │©VMware, Inc. 17
The right stuff
Do you have the expertise to best harness AI?
No 13%
Yes 87%
Recruiting the right skill
sets hinders progress
at my organization7.
78%
of organizations say that
hiring data scientists is
difficult.1
48%
of organizations say hiring
data scientists is become
more difficult over the past
3 years.1
20% Of data professionals switched jobs over
the last 12 months, (vs. 6% average)651%
Of data professionals who
switched jobs saw a salary
increase greater than 20%6.
Confidential │©VMware, Inc. 18
Cloud Cost Confusion
The resources AI needs, but how do you control and forecast spending?
It’s complicated. Is your data where it
needs to be?
Fast innovation.
New products. New
prices.
Unpredictable usage
patterns.
Humans.
Confidential │©VMware, Inc. 19
ESG
AI decisions have an impact
Does your model
discriminate?
Do the model abuse
data privacy?
Do customers have
recourse following
model decisions?
Does your model
consume renewable
energy?
What are your
model’s emissions?
Are you using the
most efficient
hardware?
Are you retraining AI-
impacted employees?
Does your model
produce safe outputs?
Can your model
improve people’s
lives?
E$G
Confidential │©VMware, Inc. 20
Are your models resilient?
Regulators expect it to be.
Can you access your
business data? Is your
backup location
synced?
Are your information
and models secure?
Are the model’s
operational
dependencies able to
overcome an outage?
Can you run your
mission-critical models
off-cloud?
Confidential │©VMware, Inc. 21
Why Businesses Struggle
IT Operator
Data Scientists /
AI Practitioner
•Need to experiment quickly
•Complicated to scale AI
apps in production
•Ticket-based infrastructure
slows development, path to
production and therefore
time to market
•Existing infrastructure performance
is insufficient for AI apps
•Shadow-IT AI/ML silos make it
challenging to manage resources
•Enterprise-class resiliency, security,
and governance (e.g. AI Act,
DORA) is difficult
22
MLOps and VMware
to the Rescue !
Confidential │©VMware, Inc. 23
What is MLOps?
MLOps, short for Machine Learning Operations, refers to the
practices and tools used to manage the lifecycle of machine learning
models. It combines machine learning, software engineering, and
operations to enable the efficient development, deployment,
monitoring, and maintenance of machine learning models at scale.
By implementing MLOps, FS organizations
can build and maintain machine learning
models that comply with regulations,
ensuring that they operate within ethical
and legal frameworks.
Confidential │©VMware, Inc.
The Domino Enterprise MLOps Platform
Confidential │©VMware, Inc. 25
Integrated MLOps –Jointly Validated Solution
Configure and control
access to AI / ML
optimized resources
for on-demand access
Leverage vMotion
migration* and DRS
initial placement with
NVIDIA powered GPU
workloads alongside
CPU workloads.
Secure and
lifecycle manage AI
infrastructure using
familiar tools without
the need to manage
a disparate AI / ML silo
Data Scientists can
consume assigned
GPU & CPU resources
to Domino Platform,
Platforms Ops assign
GPU resource
allocations via pre-
configured profiles
GPU devices can be
dedicated, shared or
partitioned to one
or more VM / TKG
workloads
* Future capability not currently available with VCF
80
NVIDIA AI Enterprise
AI and Data Science Tools and
Frameworks
On-Demand Kubernetes
Clusters and Virtual Machines
K8S Containers Virtual Machines
Intrinsic Security and
Lifecycle Management
Live Migration and Load
Balancing
Cloud-Native Deployment Infrastructure Optimization
Management Compute Storage Networking
VMware Cloud Foundation
(including vSphere)
Accelerated Mainstream Servers
NVIDIA SmartNIC / DPU*
Network Acceleration
NVIDIA GPU
Application Acceleration
Enterprise ML Ops
On-Demand Kubernetes
Clusters and Virtual Machines
Intrinsic Security and
Lifecycle Management
Live Migration and Load
Balancing
Self-Serve, Scalable
Infrastructure
Data Science & AI Workload
Orchestration
Reproducibility &
Collaboration
Governed Utilization
of GPU Resources
26
Taking the Initiative
Confidential │©VMware, Inc. 27
Generative AI is next
Your own ChatGPT? What could possibly go wrong?
Source: OpenAI CTO Andrej Karpathy
Confidential │©VMware, Inc. 28
Think outside the cloud
Workload Repatriation is a strong direction for AI
Confidential │©VMware, Inc. 30
Responsible AI
It’s about people and process
Fairness: Ensure AI
systems do not propagate
or amplify biases and treat
all users equitably.
Transparency: Make AI
systems and their decision-
making processes
understandable and
explainable to users.
Privacy and Security:
Protect users' data and
ensure that AI systems are
secure from threats.
Accountability: Ensure
mechanisms exist to hold AI
system developers and operators
responsible for their outcomes.
Confidential │©VMware, Inc. 31
Path to Production
Too many pitfalls
AI leaders can only scale
44% of models
”MISALIGNMENT”
“IRRELEVANT”
“DATA GAPS”
“TOO SLOW”
“IMPRECISE”
“TOO EXPENSIVE”
“NO SUPPORT”
Confidential │©VMware, Inc. 32
Path to Production
Remove obstacles
Empower data scientists
to scale their models.
Scheduled Jobs
Job Launchers
Model APIs
Dashboard apps
33
In Closing…
Confidential │©VMware, Inc. 34
12345
Benefits to you, our (typically) infrastructure customers
Improved
efficiency and
agility for data
scientists,
allowing for
quicker time-to-
production and
improved
business
outcomes
Better model
performance
through
continuous
monitoring and
a flexible
scalable
platform with
access to high
value hardware
Democratized
access to high-
value hardware
for all,
improving
efficiency and
making the most
of available
resources
Supports
regulatory
compliance
through security
and operational
resilience, and
adaptable to
changing
business and
regulatory
requirements
Easily extensible
to include things
such as intrinsic
security, DR,
ransomware
protection, and
capacity
optimization,
contributing to
ESG carbon
targets and cost
savings
Confidential │©VMware, Inc. 35
This integrated MLOps platform gives
data scientists what they want and
need while keeping them safe,
secure, and compliant, thus creating
better business outcomes with
reduced risk.
36
Please take
your survey
37
Questions / Discussion
39
Thank You
Confidential │©VMware, Inc. 40
References
1. Mckinsey & Co. / QuantumBlack The state of AI in 2022—and a half decade in review
2. https://www2.deloitte.com/us/en/pages/consulting/articles/state-of-ai-
2022.html?id=us:2el:3dp:wsjspon:awa:WSJCIO:2023:WSJFY23
3. Stanford 2023 AI Index -https://aiindex.stanford.edu/report/
4. Mckinsey & Co. / QuantumBlack The state of AI in 2023 -
https://www.mckinsey.com/capabilities/quantumblack/our-insights/the-state-of-ai-in-
2023-generative-ais-breakout-year
5. Accenture -The art of AI maturity -accenture.com/us-en/insights/artificial-
intelligence/ai-maturity-and-
transformation?c=acn_glb_aimaturityfrompmediarelations_13124019&n=mrl_0622.html
6. Burtchworks -2021 Salary Increases for Data Scientists & Data Engineers When
Changing Jobs
https://www.burtchworks.com/2021/12/15/2021-salary-increases-for-data-scientists-
data-engineers-when-changing-jobs/
