Encord Blog
How to build Semantic Visual Search with ChatGPT & CLIP
Written by
Eric Landau
View more postsOpenAI’s ChatGPT and CLIP releases have revolutionised the ways in which organisations and individual contributors can ship features to their users. At Encord, we’ve focused on how the neural network (CLIP) and LLM (ChatGPT) can be combined to build an effective and powerful Semantic Visual Search.
Frederik Hvilshøj, Lead ML Engineer with a PhD in Generative AI, joins Eric Landau, CEO and Co-Founder of Encord, to provide actionable insights into how to build this function from scratch.
Here are the key resources from the webinar:
- Collaboration notebook used by Frederik
- CLIP [paper/repo]
- ChatGPT [product updates, documentation]
- Encord blog: Lessons Learned: Employing ChatGPT as an ML Engineer for a Day
- Encord blog: What is vector similarity search?
Build better ML models with Encord
Get started todayWritten by
Eric Landau
View more postsRelated blogs
From Big Data to Smart Data: How to Manage, Clean and Curate Your Visual Datasets for AI Development
Webinar Recording Acquiring a dataset is just the beginning; the real challenge lies in refining it for training a Computer Vision model. Bloated, low-quality datasets waste resources and hamper model performance. The key to effective curation? Active Learning pipelines. By employing Active Learning, teams can intelligently select data that significantly impacts the model's performance. This method focuses on the model's current needs, ensuring each data point is impactful. The result is a streamlined annotation process and a more accurate, efficient Computer Vision model. Here are the key resources from the webinar: [Guide] How to curate your data [Case Study] See how one customer increased mAP by 20% through reducing their dataset size by 35% with visual data curation
Feb 01 2024
60 M
From Data to Diamonds: Unearth the True Value of Quality Data
Bridging the chasm between ‘Just AI’ and ‘Useful AI’ can be challenging, however it’s apparent that leveraging valuable data is crucial to this. As access to data increases, computer vision teams need to produce informative and reliable training data as a priority, one approach is through developing active learning pipelines. From data curation to annotation and beyond, this webinar will provide you with the tools to implement active learning pipelines and level up your computer vision models Here are the key resources from the webinar: [Guide] How to curate your data [Case Study] How one customer improved per-class performance by 67%
Nov 17 2023
60 M
How to Fine Tune Foundation Models to Auto-Label Training Data
Foundation models, like Meta’s Segment Anything Model (SAM), have provided a host of benefits for data and ML teams looking to expedite the production of training data whilst improving the quality. This webinar walks you through how to go one step further and fine-tune foundation models, in particular Meta AI's SAM, to maximize relevance to your specific use case Here are the key resources from the webinar: Encord Active GitHub - our open source tool that allowed us to conduct our research The Google Colab Notebook used in yesterday’s session Our ML Solutions Engineer’s Fine-Tune SAM blog post
Sep 21 2023
60 M
Learning Pack: The European AI Act's Impact on AI Developers
The European Parliament recently voted to adopt the EU AI Act, marking the world’s first piece of legislation on artificial intelligence. The legislation intends to ban systems with an “unacceptable level of risk” and establish guardrails for developing and deploying AI systems into production. We've put together this learning pack to equip you with all the resources you need to understand how the AI Act will impact you, whether or not you're based in the EU. From webinar recordings to the most informative blogs, sign up now to get access to these crucial resources.
Jul 13 2023
30 M
Fireside Chat: AI for Augmented Reality in 2023 and Beyond
This edition of Encord’s Fireside Chats sees Victor Prisacariu, of the University of Oxford and Niantic, sit down with Eric Landau, Encord’s CEO and Co-Founder, to discuss recent developments in AI computer vision and machine learning. Victor’s focus currently lies in real-time Augmented Reality on mobile and wearable platforms, having co-founded 6D.ai which was later acquired by Niantic in March 2020. With his wide and varied experience of the industry, Victor touched crucial areas of AR as well as discussing his work at Niantic in-depth.
May 25 2023
3 M
How to Create Workflows in Encord
Alexandre Bonnet, Solutions Engineer at Encord, walks through our latest feature: Workflows. Workflows allow ML teams to build fully customizable and automated ML pipelines to improve the efficiency of creating high-quality training data. Workflows are easy to create: Step 1: Choose from the library of 6 components (Start, Annotate, Router, Review, Archive, Complete) Step 2: Assign Annotators and Reviewers Step 3: Create arbitrarily complex workflows
May 10 2023
5 M
Webinar: Are Visual Foundation Models (VFMs) on par with SOTA?
With Foundational Models increasing in prominence, Encord's President and Co-Founder sat down with our Lead ML Engineer to dissect Meta's new Visual Foundation Model, Segment Anything Model (SAM). After combining the model with Grounding-DINO to allow for zero-shot segmentation, the team will compare it to a SOTA Mask-RCNN model to see whether the development of SAM really is revolutionary for segmentation. You'll get insights into the following: The rise of VFMs and how they differ from standard models How SAM and Grounding-DINO compare to previous segmentation models for performance and predictions What Meta's release of DINOv2 means for Grounding-DINO + SAM Evaluating model performance using Encord Active _________________ Ulrik is the President & Co-Founder of Encord. Ulrik started his career in the Emerging Markets team at J.P. Morgan. Ulrik holds an M.S. in Computer Science from Imperial College London. In his spare time, Ulrik enjoys writing ultra-low latency software applications in C++ and enjoys experimental sushi making. Frederik is the Machine Learning Lead at Encord. He has an extensive computer vision and deep learning background and has completed a Ph.D. in Explainable Deep Learning and Generative Models at Aarhus University, and published research in Efficient Counterfactuals from Invertible Neural Networks and Back-propagation through Fréchet Inception Distance. Before his P.hD., Frederik studied for an M.Sc. in computer science while being a teaching assistant for "Introduction to databases" and "Pervasive computing and Software Architecture." Frederik enjoys spending time with his two kids in his spare time and occasionally goes for long hikes around his hometown in the west of Denmark.
May 03 2023
3 M
Using ChatGPT to Improve a Computer Vision Model | Data Dojo 2023
With the proliferation of use cases for ChatGPT, we set out to investigate whether ChatGPT could be used to make improvements in other AI systems. We tested it on a practical problem in a modality of AI in which it has not been trained on - computer vision - and reported the results. ChatGPT's suggestions achieved on average a 10.1% improvement in precision and a 34.4% improvement in recall over our random sample, using a purely data-centric metric-driven approach. Eric Landau, Encord's Co-founder and CEO, sits down with the Data-Centric AI Community to share more about the process & lessons learned! __ Eric Landau is the co-founder and CEO of Encord, an active learning platform for computer vision. Before Encord, he spent nearly a decade in high-frequency trading at DRW where he was the lead quantitative researcher on a global equity delta-one desk and put thousands of models into production. He holds a M.S. in Applied Physics from Harvard University, M.S. in Electrical Engineering, and B.S. in Physics from Stanford University.
Mar 20 2023
20 M
Lessons from the Field: Fireside chat with Luc Vincent, VP of AI at Meta
Encord's, CEO & Co-founder, Eric Landau sat down with Luc Vincent, VP of AI at Meta & also Executive Advisor at Encord - as he took us through some of the lessons learned from his career, from building Lyft's first autonomous vehicle organization, to Google's geo imagery division, and now the metaverse at Meta. Luc also shared: The challenges and lessons learned from setting up & scaling world-class computer vision orgs that are pushing the boundaries of AI The working environment and culture in the early days of Google The ML applications and promising projects he's most excited about going into the next decade And what it'll take for the winners to get ahead. Eric Landau is the co-founder and CEO of Encord, an active learning platform for computer vision. Before Encord, he spent nearly a decade in high-frequency trading at DRW where he was the lead quantitative researcher on a global equity delta-one desk and put thousands of models into production. He holds a M.S. in Applied Physics from Harvard University, M.S. in Electrical Engineering, and B.S. in Physics from Stanford University.
Feb 27 2023
4 M
The Future of ML Teams: Embracing Active Learning
Eric Landau, Co-Founder & CEO of Encord, talks about "Active Learning & the ML Team of the Future" to close the AI prototype to production gap @ AI at Scale 2023 organised by the AI Infrastructure Alliance. 💡Read more: The full guide to Active Learning in Machine Learning Eric Landau is the co-founder and CEO of Encord, an active learning platform for computer vision. Before Encord, he spent nearly a decade in high-frequency trading at DRW where he was the lead quantitative researcher on a global equity delta-one desk and put thousands of models into production. He holds an S.M. in Applied Physics from Harvard University, M.S. in Electrical Engineering, and B.S. in Physics from Stanford University.
Feb 26 2023
20 M
Synthetic Data & Generative AI: Fireside chat with Synthesia Co-Founder & CEO Victor Riparbelli
Founded in 2017, Synthesia is one of the early pioneers of generative AI and the first to synthesize video from text. Synthesia has grown to a team of 135 employees and 15,000+ customers. Among its clients are fast-food giants including McDonald’s, research company Teleperformance and global advertising holding company WPP. In this fireside chat, Eric Landau, Co-Founder & CEO of Encord, will talk to Victor Riparbelli, Co-Founder & CEO of Synthesia, about what the early days of Synthesia looked like, the current & future role of synthetic media, and setting up & scaling a world-leading research organisation pushing the boundaries of synthetic data generation.
Feb 26 2023
55 M
Software To Help You Turn Your Data Into AI
Forget fragmented workflows, annotation tools, and Notebooks for building AI applications. Encord Data Engine accelerates every step of taking your model into production.