Techcelerate AI Product Workshop delivered by Rhys Jones on 14th of Oct 2014 at Sci Tech Daresbury
https://techcelerate.ventures
https://axonzeta.ai
About Techcelerate Ventures
Tech Investment and Growth Advisory for Series A in the UK, operating in £150k to £5m investment market, working with #SaaS #FinTech #HealthTech #MarketPlaces and #PropTech companies.
Transform your
existing products
with GenAI
Rhys Jones
rhys@axonzeta.ai
Monday 14th October 2024
GenAI
● Background & Trends
● Building and Supporting Products
● Improving Product UX and Workflows
● Demos & Code
● Discussion
GenAI Growth
GenAI Providers
Transformers vs other paradigms in Foundation Models
stateof.ai 2024
https://www.youtube.com/watch?v=zjkBMFhNj_g
GenAI Training
Karpathy: https://www.youtube.com/watch?v=zjkBMFhNj_g
GenAI Performance
stateof.ai 2024
2023
100x drop!
2024
GenAI Costs
60x drop!
2023
2024
stateof.ai 2024
GenAI Open Models
stateof.ai 2024
https://www.youtube.com/watch?v=zjkBMFhNj_g
GenAI : NN Predicting Next Word
Karpathy: https://www.youtube.com/watch?v=zjkBMFhNj_g
GenAI
● Background & Trends
● Building and Supporting Products
● Improving Product UX and Workflows
● Demos & Code
● Discussion
Building and Supporting Products
● Software Engineering
● Support
● Sales
● Marketing
ICONIQ: State of AI 2024
ICONIQ: State of AI 2024
ICONIQ: State of AI 2024
ICONIQ: State of AI 2024
GenAI Tools
● Replit: AI Coding
● NotebookML : Research, Q&A,(Podcasts!)
● NapkinAI : Documentation, Diagrams
● DifyAI : Custom Support Agents
● SciPhi : RAG Tools
● Synthesia : Product Video Generation
● BrainBox / ChatGPT : of course…
GenAI
● Background & Trends
● Building and Supporting Products
● Improving Product UX and Workflows
● Demos & Code
● Discussion
Improving Product UX and Workflows
Improving Product UX and Workflows
Discoverability & Search
● Semantic Search
● Vector Stores
● RAG : Retrieval Augmented Generation
● Chatbot Assistants for your Product
https://towardsdatascience.com/a-guide-to-word-embeddings-8a23817ab60f
Discoverability & Search
https://pub.towardsai.net/introduction-to-retrieval-augmented-generation-rag-using-langchain-and-lamaindex-bd0047628e2a
Discoverability & Search
https://pub.towardsai.net/introduction-to-retrieval-augmented-generation-rag-using-langchain-and-lamaindex-bd0047628e2a
Discoverability & Search
https://udify.app/chat/veXxQpTYGdMHv9Fj
Discoverability & Search
https://udify.app/chat/veXxQpTYGdMHv9Fj
Discoverability & Search
Improving Product UX and Workflows
ChatGPT Approach
Agentic Approach
https://dify.ai/
Agentic Approach : Platforms
https://www.baeldung.com/java-langchain-basics
Agentic Approach : LangChain
Agentic Approach
Agentic Approach
Agentic Structured Outputs
Agentic Structured Outputs
https://www.jasondonovan.com/
Agentic Approach
● Takes any unstructured data
● Applies GenAI task
● Outputs correctly
structured data
Agentic Approach
https://blog.ml6.eu/what-you-didnt-want-to-know-about-prompt-injections-in-llms-4579db1794
Safety and Security
● Function Calling by Agents
● How safe are inputs?
● Prompt Injection
● Restrict Function scopes
● Supervisor Agents
● Use JSON not Functions
GenAI
● Background & Trends
● Building and Supporting Products
● Improving Product UX and Workflows
● Demos & Code
● Discussion
Frontend UX
Frontend UX
Frontend UX
Frontend UX
https://py-todo-web-axonzeta.replit.app/
Frontend UX
https://py-todo-web-axonzeta.replit.app/
Backend Workflows
Backend Workflows
https://ab-test-analyzer-axonzeta.replit.app/
Backend Workflows
https://ab-test-analyzer-axonzeta.replit.app/
Multi Agent Systems
● Agents work well with other Agents
● Some Agents DO others DECIDE and ROUTE
Multi Agent System Example
Multi Agent System Example
Multi Agent System Example
0161 524 9240
GenAI
● Background & Trends
● Building and Supporting Products
● Improving Product UX and Workflows
● Demos & Code
● Discussion
Agenda
09:00 Registration and networking
09:30 Introduction to Techcelerate by Manoj
09:45 Transform your existing product by adding GenAI to automate user workflows, improve features and enhance UX
(beyond just chatbots!) - by Rhys
10:30 Interactive discussion with Techcelerate companies to understand and explore how they plan on adding AI to their
products - Rhys and facilitated by Manoj
11:00 Learn about AxonZeta and Google Chrome's new on-device AI, giving you a cost-free, easy-to-use LLM runtime for
your Web Apps - by Rhys
11:30 Breaking the barriers in your tech startup journey - by Manoj
12:00 Networking
Discussion
Agenda
09:00 Registration and networking
09:30 Introduction to Techcelerate by Manoj
09:45 Transform your existing product by adding GenAI to automate user workflows, improve features and enhance UX
(beyond just chatbots!) - by Rhys
10:30 Interactive discussion with Techcelerate companies to understand and explore how they plan on adding AI to their
products - Rhys and facilitated by Manoj
11:00 Learn about AxonZeta and Google Chrome's new on-device AI, giving you a cost-free, easy-to-use LLM runtime for
your Web Apps - by Rhys
11:30 Breaking the barriers in your tech startup journey - by Manoj
12:00 Networking
Run your own AI
with local SLMs
Google Chrome | AxonZeta
LLMs and SLMs
● Large Language Models
○ Typically > 70B parameters
○ GPT4 estimated to be 1,760B params
● Small Language Models
○ Typically < 20B parameters
○ Quantized to e.g. 4 Bits per param
https://huggingface.co/spaces/open-llm-leaderboard/open_llm_leaderboard
Small Model Performance
Phi 3.5
3.8B parameters
~ 7.7GB memory
Quantized
to 4 bits / param
~ 2GB memory
https://laptopmedia.com/comparisons/apple-m4-vs-m3-pro-max-vs-m2-pro-max-ultra-vs-m1-pro-max-ultra-the-ultimate-benchmark-comparison/
Laptop Performance
On-Device Small Models
● Local processing of Sensitive Data
● Snappy user experience
● Full control of AI Models
● Offline AI use
● Change in economic model
○ User already has the resource
○ No extra “$20/month for our AI!”
On Device SML Benefits
In-Browser AI
https://developer.chrome.com/docs/ai/built-in
In-Browser AI
Google AI Studio
Google Vertex AI
[AZ Product]
Cross Device
Custom Models
[AZ Product]
● Development platform for On-Device SML AI
● Prompting & Training
● Evaluation & Monitoring
● Leverage multiple SLMs
● Works across all Browsers
[AZ Product]
● Add local AI Agents to web apps
● Test with different Small Models
● Promotes private, free, local AI
● User still in charge
● Combine with Hybrid on-cloud or on-prem
https://docs.google.com/file/d/1EtNLwnLobiJN3jJ6mMCFvYqFTZU_MRMi/preview
https://docs.google.com/file/d/1270oVY6sXshwzi4wgJJUoO0aA1jQzI11/preview
[AZ Product]
Run your own AI
with local SLMs
Google Chrome | AxonZeta
Agenda
09:00 Registration and networking
09:30 Introduction to Techcelerate by Manoj
09:45 Transform your existing product by adding GenAI to automate user workflows, improve features and enhance UX
(beyond just chatbots!) - by Rhys
10:30 Interactive discussion with Techcelerate companies to understand and explore how they plan on adding AI to their
products - Rhys and facilitated by Manoj
11:00 Learn about AxonZeta and Google Chrome's new on-device AI, giving you a cost-free, easy-to-use LLM runtime for
your Web Apps - by Rhys
11:30 Breaking the barriers in your tech startup journey - by Manoj
12:00 Networking
Networking
existing products
with GenAI
Rhys Jones
rhys@axonzeta.ai
Monday 14th October 2024
GenAI
● Background & Trends
● Building and Supporting Products
● Improving Product UX and Workflows
● Demos & Code
● Discussion
GenAI Growth
GenAI Providers
Transformers vs other paradigms in Foundation Models
stateof.ai 2024
https://www.youtube.com/watch?v=zjkBMFhNj_g
GenAI Training
Karpathy: https://www.youtube.com/watch?v=zjkBMFhNj_g
GenAI Performance
stateof.ai 2024
2023
100x drop!
2024
GenAI Costs
60x drop!
2023
2024
stateof.ai 2024
GenAI Open Models
stateof.ai 2024
https://www.youtube.com/watch?v=zjkBMFhNj_g
GenAI : NN Predicting Next Word
Karpathy: https://www.youtube.com/watch?v=zjkBMFhNj_g
GenAI
● Background & Trends
● Building and Supporting Products
● Improving Product UX and Workflows
● Demos & Code
● Discussion
Building and Supporting Products
● Software Engineering
● Support
● Sales
● Marketing
ICONIQ: State of AI 2024
ICONIQ: State of AI 2024
ICONIQ: State of AI 2024
ICONIQ: State of AI 2024
GenAI Tools
● Replit: AI Coding
● NotebookML : Research, Q&A,(Podcasts!)
● NapkinAI : Documentation, Diagrams
● DifyAI : Custom Support Agents
● SciPhi : RAG Tools
● Synthesia : Product Video Generation
● BrainBox / ChatGPT : of course…
GenAI
● Background & Trends
● Building and Supporting Products
● Improving Product UX and Workflows
● Demos & Code
● Discussion
Improving Product UX and Workflows
Improving Product UX and Workflows
Discoverability & Search
● Semantic Search
● Vector Stores
● RAG : Retrieval Augmented Generation
● Chatbot Assistants for your Product
https://towardsdatascience.com/a-guide-to-word-embeddings-8a23817ab60f
Discoverability & Search
https://pub.towardsai.net/introduction-to-retrieval-augmented-generation-rag-using-langchain-and-lamaindex-bd0047628e2a
Discoverability & Search
https://pub.towardsai.net/introduction-to-retrieval-augmented-generation-rag-using-langchain-and-lamaindex-bd0047628e2a
Discoverability & Search
https://udify.app/chat/veXxQpTYGdMHv9Fj
Discoverability & Search
https://udify.app/chat/veXxQpTYGdMHv9Fj
Discoverability & Search
Improving Product UX and Workflows
ChatGPT Approach
Agentic Approach
https://dify.ai/
Agentic Approach : Platforms
https://www.baeldung.com/java-langchain-basics
Agentic Approach : LangChain
Agentic Approach
Agentic Approach
Agentic Structured Outputs
Agentic Structured Outputs
https://www.jasondonovan.com/
Agentic Approach
● Takes any unstructured data
● Applies GenAI task
● Outputs correctly
structured data
Agentic Approach
https://blog.ml6.eu/what-you-didnt-want-to-know-about-prompt-injections-in-llms-4579db1794
Safety and Security
● Function Calling by Agents
● How safe are inputs?
● Prompt Injection
● Restrict Function scopes
● Supervisor Agents
● Use JSON not Functions
GenAI
● Background & Trends
● Building and Supporting Products
● Improving Product UX and Workflows
● Demos & Code
● Discussion
Frontend UX
Frontend UX
Frontend UX
Frontend UX
https://py-todo-web-axonzeta.replit.app/
Frontend UX
https://py-todo-web-axonzeta.replit.app/
Backend Workflows
Backend Workflows
https://ab-test-analyzer-axonzeta.replit.app/
Backend Workflows
https://ab-test-analyzer-axonzeta.replit.app/
Multi Agent Systems
● Agents work well with other Agents
● Some Agents DO others DECIDE and ROUTE
Multi Agent System Example
Multi Agent System Example
Multi Agent System Example
0161 524 9240
GenAI
● Background & Trends
● Building and Supporting Products
● Improving Product UX and Workflows
● Demos & Code
● Discussion
Agenda
09:00 Registration and networking
09:30 Introduction to Techcelerate by Manoj
09:45 Transform your existing product by adding GenAI to automate user workflows, improve features and enhance UX
(beyond just chatbots!) - by Rhys
10:30 Interactive discussion with Techcelerate companies to understand and explore how they plan on adding AI to their
products - Rhys and facilitated by Manoj
11:00 Learn about AxonZeta and Google Chrome's new on-device AI, giving you a cost-free, easy-to-use LLM runtime for
your Web Apps - by Rhys
11:30 Breaking the barriers in your tech startup journey - by Manoj
12:00 Networking
Discussion
Agenda
09:00 Registration and networking
09:30 Introduction to Techcelerate by Manoj
09:45 Transform your existing product by adding GenAI to automate user workflows, improve features and enhance UX
(beyond just chatbots!) - by Rhys
10:30 Interactive discussion with Techcelerate companies to understand and explore how they plan on adding AI to their
products - Rhys and facilitated by Manoj
11:00 Learn about AxonZeta and Google Chrome's new on-device AI, giving you a cost-free, easy-to-use LLM runtime for
your Web Apps - by Rhys
11:30 Breaking the barriers in your tech startup journey - by Manoj
12:00 Networking
Run your own AI
with local SLMs
Google Chrome | AxonZeta
LLMs and SLMs
● Large Language Models
○ Typically > 70B parameters
○ GPT4 estimated to be 1,760B params
● Small Language Models
○ Typically < 20B parameters
○ Quantized to e.g. 4 Bits per param
https://huggingface.co/spaces/open-llm-leaderboard/open_llm_leaderboard
Small Model Performance
Phi 3.5
3.8B parameters
~ 7.7GB memory
Quantized
to 4 bits / param
~ 2GB memory
https://laptopmedia.com/comparisons/apple-m4-vs-m3-pro-max-vs-m2-pro-max-ultra-vs-m1-pro-max-ultra-the-ultimate-benchmark-comparison/
Laptop Performance
On-Device Small Models
● Local processing of Sensitive Data
● Snappy user experience
● Full control of AI Models
● Offline AI use
● Change in economic model
○ User already has the resource
○ No extra “$20/month for our AI!”
On Device SML Benefits
In-Browser AI
https://developer.chrome.com/docs/ai/built-in
In-Browser AI
Google AI Studio
Google Vertex AI
[AZ Product]
Cross Device
Custom Models
[AZ Product]
● Development platform for On-Device SML AI
● Prompting & Training
● Evaluation & Monitoring
● Leverage multiple SLMs
● Works across all Browsers
[AZ Product]
● Add local AI Agents to web apps
● Test with different Small Models
● Promotes private, free, local AI
● User still in charge
● Combine with Hybrid on-cloud or on-prem
https://docs.google.com/file/d/1EtNLwnLobiJN3jJ6mMCFvYqFTZU_MRMi/preview
https://docs.google.com/file/d/1270oVY6sXshwzi4wgJJUoO0aA1jQzI11/preview
[AZ Product]
Run your own AI
with local SLMs
Google Chrome | AxonZeta
Agenda
09:00 Registration and networking
09:30 Introduction to Techcelerate by Manoj
09:45 Transform your existing product by adding GenAI to automate user workflows, improve features and enhance UX
(beyond just chatbots!) - by Rhys
10:30 Interactive discussion with Techcelerate companies to understand and explore how they plan on adding AI to their
products - Rhys and facilitated by Manoj
11:00 Learn about AxonZeta and Google Chrome's new on-device AI, giving you a cost-free, easy-to-use LLM runtime for
your Web Apps - by Rhys
11:30 Breaking the barriers in your tech startup journey - by Manoj
12:00 Networking
Networking