Magical Transcripts using AI on Mindstone

Introduction

So I'm Florian, I'm CTO and co-founder here at MindStone. We founded the company four years ago.

Growing Video Content on the Platform

And recently we had a lot more videos that have been uploaded on our platform due to the meetups. So we can see an example of that here.

Challenges with the Original Transcript

And the video has like a transcript below, which we didn't really like. It's hard to read. It feels like a wall of text. So we try to improve that.

Uploading and Processing New Transcripts

So I'm going to upload the same talk now from my desktop. I downloaded that earlier. And it's going to take a few minutes to process the file, but it's going to generate a new transcript.

Improving the Transcription Process

So before that, we were using just Whisper, the basic model from OpenAI. And we decided to change that and improve on it.

Seeking a Faster Transcription Solution

So the first step was finding a faster Whisper. The version we had was quite slow. It was taking up to five minutes for a 15 minutes video, I think. So that was really, really slow.

Incorporating WhisperX and Wave2Vec

So we found something called WhisperX, which runs the model a bit faster. And we also use Wave2Vec to get the transcript by exact sentences. From there, it's great.

Enhancing Readability with GPT

We have all the sentences, but we wanted to make it look more like an article. So what we decided to do is actually use GPT for that.

Finalizing the Transcript

So we asked ChatGPT to create some headings into our transcript, which allows us to group information together. Then we asked him to create paragraphs from the sentences to make it look like an article. And then we redo the title because we realized that ChatGPT was doing a better job the second time when the data was already in paragraph.

And then after that, we add everything that we need for the annotations that we can generate. And hopefully, within a few seconds, it should get there. But we should see the new transcript appear. There we go.

Results and Comparison

So we can see now, so if we compare from the previous one, where we just have a wall of text, it now has created an introduction, explain what Agents Runner is. You can actually read the article without going through the video or going through the podcast, because that will work as well with podcasts, YouTube videos, or anything that is with audio.

Maintaining Previous Functionalities

And we still have the same functionality as before. So you can't hear the sound, I think. But there is like a karaoke mode going as well. So you can follow the video and the text at the same time.

Conclusion

And yeah, that's the demo for today.

Invitation to Try MindStone

So please try. You can upload your own video. You can upload your YouTube videos as well or any podcast that you have. And if you want to annotate them, you can just add them to Mindstorm.

Finished reading?