Artwork

Player FM - Podcast-app
Gå offline med appen Player FM !

AI for Course Creators with Aaron Edwards From DocsBot AI

47:45
 
Del
 

In this LMScast episode, Aaron talks with Chris Badgett about AI for course creators. He shares the journey of developing DocsBot.ai, from creating a proof-of-concept chatbot trained on WordPress documentation to building a powerful tool that enables companies to leverage their own documentation for content creation, customer service, and more.

Aaron Edwards is the creator of DocsBot.ai, a cutting-edge chatbot driven by AI that improves document management and business assistance. Aaron explains the fundamental idea of retrieval-augmented generation (RAG), which blends generative.

Image of Aaron Edwards

By giving the AI context, real-world data can help it provide precise, well-founded responses. This method tackles a prevalent issue with AI, which is that it “hallucinates” or fabricates knowledge. By giving the AI context, real-world data can help it provide precise, well-founded responses. This method tackles a prevalent issue with AI, which is that it “hallucinates” or fabricates knowledge.

Data from the actual world can assist the AI give accurate and well-founded answers by providing context. A common problem with AI is that it “hallucinates” or creates knowledge, which this approach addresses.

This podcast offers insightful information on the technological foundations of DocsBot.ai and how it can be used to many businesses, from agencies to course developers, to enhance assistance, productivity, and content reuse.

Here’s Where To Go Next…

Get the Course Creator Starter Kit to help you (or your client) create, launch, and scale a high-value online learning website.

Also visit the creators of the LMScast podcast over at LifterLMS, the world’s leading most customizable learning management system software for WordPress. Create courses, coaching programs, online schools, and more with LifterLMS.

Browse more recent episodes of the LMScast podcast here or explore the entire back catalog since 2014.

And be sure to subscribe to get new podcast episodes delivered to your inbox every week.

2023 WordPress LMS Buyer’s Guide

Exclusive Download! Stop wasting time and money researching online course and membership site tech.

WordPress LMS Buyer's Guide Download Cover Images

Episode Transcript

Chris Badgett: You’ve come to the right place if you’re looking to create, launch, and scale a high value online training program. I’m your guide, Chris Badgett. I’m the co-founder of LifterLMS, the most powerful learning management system for WordPress. State of the end, I’ve got something special for you. Enjoy the show.

Hello and welcome back to another episode of LMScast. I’m joined by my friend Aaron Edwards. He’s from DocsBot. ai. That’s DocsBot. ai. Go check this out. This is not going to be your average episode where we talk about chat GVT or some basic AI concepts, although we are going to talk about that stuff some.

We’re going to go deep. Aaron has been deep in the weeds with WordPress, AI, software development, solving problems for business owners. We’re going to get into it today. This is going to be a fascinating episode, but first welcome back on the show, Aaron.

Aaron Edwards: Thank you for having me, Chris.

Chris Badgett: Good to see you again.

I always love running into you because you’re always up to like really interesting, cutting edge things. Whether it was NFT stuff in the past, solving challenges. With the media library and uploads and stuff like that. I’ve watched you over the past couple, maybe several years launched DocSpot.

So we’re going to focus on the AI component both for WordPress professionals and agencies, but also for course creators. But first, just at a high level, what is DocSpot. AI?

Aaron Edwards: Yeah, sure. The idea is let me go back. I’ll tell my story a little bit with it. As you said, I just love playing with the latest new technology.

So obviously as this generative AI stuff started coming to the forefront I was just diving in and building things, playing around with it, learning the latest Tech and stuff like that. There was images and I know we talked about in the previous episode and now doc spot and what I started out is this new technique called retrieval augmented generation.

So, chat GPT dropped in what is end of 2022, I think. And it like wowed everyone, obviously, like we’re like, Oh my gosh. But people quickly realized that what hallucinations were. So these AI models are trained to be like super, super helpful and great assistance so much so that they’ll just make stuff up to make you happy. There’s a new in that time during that fall, there’s a new technique some papers and things that have come out where People had realized that, okay, if you provide like chat GPT with context, for example, you paste in a page of text or whatever about your product or something like that you paste that in there and then you ask a question about that content that you pasted in.

Now it has like facts to ground its answers. So that way it can actually like. Provide like a accurate and valid answer by summarizing or extracting information from that text that you pasted in this context. So that’s what. This new technique called retrieval augmented generation is.

And so, I just was playing with that and I thought this would be really cool just as a proof of concept. And to learn myself is to build a chat bot that was trained on all the WordPress documentation. So I went and I like built like a web crawler that went and crawled all the documentation pages on wordpress.

org. Whether it’s the developer side or like the user kind of side, all that kind of stuff. And then. into a, it’s called a vector database, which is just a way to store that training data using AI embeddings it’s called, which is a way to do semantic search so that if you search for you give a question and it’s able to identify texts that may not have the exact same words in the question, but I do.

but able to identify pieces of text that are the same like semantic meaning or close semantic meaning. So basically that way you can transform a user’s question into an actual whatever the most relevant document pieces of documentation are to answer that question. And so I launched that and it was called, um, chat WP and it’s still online.

It’s a, let me see if I remember the URL even as WP docs dot chat. And that’s still like a free tool that’s out there. Ask me anything about WordPress. And so the kind of ideal of that was, actually I’ll share my screen. Is that all right?

Chris Badgett: Yeah, and if you’re listening to this on the podcast, just head on over to the Lifter LMS YouTube channel and look for Aaron Edwards, and you’ll find what we’re looking at here.

Aaron Edwards: Yeah, so wpdocs. chat. Just a simple chat interface, where you could ask like questions about WordPress and then it goes. And then from that training data, it’s able to provide like. Output and examples and things like that, even code snippets and stuff that is learned from the documentation.

So that was my my learning experience and proof of concepts. And then what I did is people like were super excited about it. So I slapped a little a waitlist signup form at the bottom of the page. And I said, Hey, if you’re interested in this for your business. Let me know. And I just put a little survey there what are like the key features that you need, that kind of thing.

If you’re trying to use this for your own business and I got a lot of like good responses on that, like a good wait list and the thought, all right, this is like somewhat proven that there’s some desire, for businesses to have something like this. And I spent a month working nights all night, to put out duck spot, so that was the product, which has been going well.

It’s grown a lot, it’s my full time job now. It’s been More than a year and a half. And basically what we do is we make it super easy for businesses to train their support. It can be used internally, like for your team to be able to get answers quickly from your existing documentation, files SOPs, that kind of thing it can be used.

Some of our customers use it. For, the classic content generation or repurposing. So they have their chatbot trained with all their marketing content or whatever. And they can use that to generate, ad copy or different things like that. Or if their agency, they might have a chatbot trained for each of their clients.

And they’ll use that for content repurposing a lot of different use cases.

Chris Badgett: So what was the surprise to you as people started using it? Like who was like the main group and then what were some surprising use cases that started popping up?

Aaron Edwards: Yeah. I think the biggest thing, and this is really what led it to it being a success is somehow some Japanese Twitter tech influencer.

Like from my chat bot, like in Japanese and just shared a video of it, answering questions in Japanese. And that went totally viral in Japan. It was so crazy. I like woke up in the morning and suddenly. I had 100 support tickets in Japanese and my servers are crashing. And I was getting all these signups and things like that.

I am really stressed testing it. So it’s like, What is going on? Yeah, so it went viral in Japan, which is pretty crazy. So still probably maybe a third of my customers are Japanese. And that really led me down the path of doing the work that kind of sets us apart from a lot of our competitors and that making sure that it’s very like multilingual and compatible with non English languages.

Most of all the AI announcements and benchmarks and all that stuff that you see. online and all the hype. It’s all English. Like you don’t actually realize it that, that, okay. Yeah. This tiny model that’s like amazing that like Lama or whatever that Lama three or whatever, that meta put out it sounds like so amazing, but you realize, Oh, it’s only trained on The four most common like Western languages, and the benchmarks suck on other languages.

So that’s been a really big thing, like from prompts optimization on the backend to making sure it like can detect languages properly and answer and whatever the language that the user asked questions, that kind of thing. It’s taken a lot of like work and even ongoing as I add new features, making sure that It works for all those use cases,

Chris Badgett: tell us more about like the training data or the source pages.

So for example, if I’m an agency, I may have a bunch of case studies and service offerings and a blog, bunch of blog posts about how I approach my agency work or my clients, or podcaster and I have a lot of podcast episodes. Or I’m a, I have a product that has support library of content. Like, how do people think about source pages and what could be sources?

And at what point, how much source content do you need for this to get really valuable?

Aaron Edwards: Yeah. That’s like one of the harder things that we do is like the actual like code to create these like chatbots and do the retrieval augmented generation, and that’s all open source. There’s all kinds of.

Like things that you can do and probably spend that up. If you’re a developer and just a few hours, something that kind of working as a proof of concept, but really the hard part is like figuring out how to take like real world data that businesses have. Whether it’s course content, as you mentioned, documentation, crawling a website, all that kind of thing and transforming that into something that the bot understands and is able to use to answer questions accurately, without hallucinations.

So we developed a lot of different,

Source pages or ways to, Import information to your bot. So whether it’s like uploading document files PDFs or docs, PowerPoint, Excel, whatever it is, each of those things, we have to have different logic for how we parse that and how we divide it up into pieces.

That that the LLM, the large language model AI can understand. So there’s a lot of work in that. You can train it just with like FAQs that are simple to answer. You can have a crawl your entire website, like sitemap, that kind of thing. And connected to tons of cloud sources, to your notion, confluence, Salesforce, get book places where you already have documentation stored, or if you’re using like Zen desk or help scout or intercom fresh desk, like a lot of these popular, like support tools that maybe you have your knowledge base in, or you have your previous Customer support, history tickets in there, and you can use all that to train your bot so that it can answer questions into the future.

So we’re always adding like new data sources. A fun one that I worked on last month is YouTube. Yeah, it sounds awesome. You’re mentioning that you have all these YouTube videos, right? , so being able to train your bot just by dropping in a YouTube video, URL or like a URL to a playlist of a.

Of, up to a hundred different videos. And then it goes through and actually had to, this is the hard part. YouTube does not like scraping. So it’s literally like people don’t realize that’s the hard part. I had to use like a whole network of residential proxies, so it’s actually going through like people’s home computer to scrape the subtitles from YouTube and then use that to train the bot so that you can chat with your YouTube videos.

And and even as you’re chatting with them, it like provides source links to where it got the answer. So it’ll actually link to the correct second in the video where it pulled that answer from which is fun.

Chris Badgett: Very cool. In terms of humans. Potentially being concerned about I don’t want this AI to speak for me, like how what are the options in terms of just cut it loose and let it talk to anybody versus a human reviews before it gets communicated?

Like, how do you think about that moderation aspect?

Aaron Edwards: Yeah, I think and a lot of our customers. They’re using it for frontline customer support. So tier one. So if you think about it, first of all, if you have a business, that customers never read the documentation, right? A

Chris Badgett: lot of our support is here’s the link to the doc.

Exactly.

Aaron Edwards: Yes. That’s whatever, 75 or something percent of the average company support is like here’s the link to the document where they answer that question for you, and in the, Previously, there’d be things like search tools and stuff try to identify that to try to prevent tickets being created, deflection, we call it but why not have the same thing that Google does now when you Google something and it gives an AI answer by looking at the top results and summarizing that into a succinct answer instead of having to go to all these pages and read and figure out how to answer your question.

That’s what we do. You take all the existing information that you’ve already written at some point, that’s already been recorded at some point in your documentation or support history or whatever, or YouTube demo videos, whatever it may be. And provide like an instant, like quick answer to them.

That’s grounded in that truth of your training data and also provides links to where it got that information. So they could click out and. And verify that if they want to or whatever. So yeah, that’s one side of it. And I think that these models are definitely well proven to be good enough for that kind of tier one support.

And I’m surprised like what it can answer too. So even like my bot is trained on all my like developer documentation for APIs and things like that. So I find myself like asking questions about how do I give me an example code for how I integrate my widget with Zendesk, And it literally both reads like my documentation, but it’s actually like performing logic and reasoning as well after reading that, just like a human would to generate new answers.

It’s not just copying, copying and pasting verbatim which is pretty amazing that these AI models are capable of doing that reasoning. So that even gets you more than what you’d say is the tier one support, because it’s not just. Not just linking to documentation, but it’s actually able to do some reasoning to

Chris Badgett: that’s very cool.

I know one of the things that I think about when, looking at AI tools is cost and we don’t have to get into the specifics of your pricing because I know it might change. But As the better AI gets, it might require more resources, which could in theory, make it more expensive. Or you need a large training set.

Like, how do you think about pricing and making it work for everybody? The the business owner, you making sure it’s a profitable venture. Like, how do you think about that?

Aaron Edwards: Man, when I launched the same week that the. Chat GPT API became available which was great timing, right? But the costs for that, like even the old, like 3.

5 model, chat, GPT model it would cost, I think three or four cents per question per user question. Now, if you’re getting like 10, 000 questions a month or

something that adds up fast, right? But at the same time, if you compare that to hiring a support person, the answer, those Whatever, 70 percent of those 10, 000 questions. That’s still super cheap, right? It was hard thing to, to factor in. And so what I actually did in the beginning is we did bring your own API key.

So you could actually put in your key from open AI, like into our dashboard and we’d save it like encrypted and use that At cost. So instead of reselling credits or something like that, like some of our competitors do with a markup, we just basically at cost, for the actual processing.

And what’s amazing is in, in less than two years, the The price for how much power you get, like from the AI model has literally dropped more than a hundred times. It’s yeah, it’s been crazy. And as a business who’s like trying to figure out like my pricing models and things like that, and how to charge.

That’s been great both for my customers who are like bringing their own key, but now I’m able to provide access like included to make it simpler for people to get started. They don’t have to have their own key, so like I heard, I think it was

said that if you’re building products on AI, like it’s changing so fast and its capabilities are changing so fast, you need to be building like on the cutting edge to where right now. It’s seems super expensive or super like impossible for it to do. Because it’s just change every two to three months.

It’s like a new model comes out two to three times price drop. It’s amazing.

Chris Badgett: Help us understand like the models in the background are you leveraging open AI or you’re leveraging llama, a combination of stuff? Like how does it, like what powers all this?

Aaron Edwards: Yeah, right now we’re using only a open AI models just cause they have the best, like developer ecosystem and things like that.

We are playing with other models like Gemini and Llama. Maybe, like I said, there’s still weaknesses there for other languages. So that’s one of the things that we have to test heavily is we want to support having a chat bot in Japanese, right? And that it’s going to be able to, no matter what language the user asks a question, that it’s able to still find the right documentation, even though maybe your documentation is in English.

If someone asks a question in Japanese, it should be able to use AI to find the relevant documentation and provide an accurate answer to the user. And so for that, you need some of these larger models that are, have much more training data from other languages than English. So right now that’s the GPT 4.

0. That’s what kind of like our workhorses. So food like for free, like along with your plan. And that works for almost all use cases and it’s super fast and it’s super good. Multilingual as well. And then if you like, you want Much more advanced reasoning, like to where it can maybe do some custom coding and things like that as part of its answers.

Then you could jump up to the full, like GPT 4. 0 model, which has dropped down in price substantially to, it still costs less than one cent per question, more like 0. 5 really. So super cost effective. And they just keep getting more and more powerful.

Chris Badgett: Could you describe to the layman, just to highlight the rate of change with AI what’s the difference with 4.

0 in OpenAI versus 3. 5 or Turbo or whatever came before?

Aaron Edwards: Yeah a lot of it’s like proprietary, so they don’t tell us like all the technical details you can infer. There’s been a lot of new inventions. So first of all, was just like the amount of data is trained on, right? Like call that parameters, like how many parameters, that’s the size of the model.

And so like you had Like GPT two, I think. And then GPT three which is like the first ones that they had like an API. If you were really lucky, you could get a key to use that one. And that’s what some of the early like content creation things, um, used and then 3. 5 was basically, they took that same model and they did what’s called RLHF, which is basically they they Took had humans, right?

A bunch of okay, here’s the message. And then here’s what a good response is for that. And so they hired, armies of people and in Africa and other third world countries that speak English and other data set online to do that reinforcement learning on top of the existing model, which is basically where the invention of chat GPT came from.

So instead of being something where you had to figure out these weird Ways of prompting it crazily to where it like completes, like what you started. Now you can just give any kind of instruction and it just knows how to follow that instruction and provide a good output. So that didn’t really change the size of the model.

It just made it much more useful to the average user. And then you had the next big jump in model size and that was GPD four. And they say that went from GPD three is probably something like four or 500 Billion parameters. I think they estimate maybe three or 400 billion parameters.

And GPD four is probably like closer to 2 trillion. Wow. Yeah. And that’s after being pruned and cleaned up and that kind of stuff. It’s more like a tenfold increase, in size. And so that’s why it’s so much more expensive and slower. Cause it’s got to run on a whole cluster of 20, 000 GPUs, because it uses so much memory.

To do all that kind of stuff. And then I’ll put on top of that, then they’re able to do what’s called distillation, which is where you take that big model and you pull out by using like the reinforcement learning and things like that training, what’s the most useful. Parts to you, so it may not have the widespread knowledge from all that training that pulled in, but all the things that people most commonly ask, or most commonly use it for logic, it kept only those parts so you can have a 10 times smaller model that was trained that was distilled from the big one.

I’m just pulling out the most useless things. And that’s what GPT 40 mini is which is like their most cost effective model. It’s super, super fast, super inexpensive, but still very powerful, even in other languages.

Chris Badgett: That’s awesome. And Aaron, while I’ve got you an AI expert and who’s great at teaching and communicating this stuff.

Help me understand the buzz around two concepts. I’m not sure if they’re related or not, but one is having a large context window and the other is chain of thought reasoning. What are those?

Aaron Edwards: Okay. They’re not like necessarily related, but the context window is basically like How much text or video or whatever the input, how much the AI model can handle at once.

Chris Badgett: If you’re using chat, GBT, sometimes you’re like, you try to copy paste and it’s too long or whatever.

Aaron Edwards: Yeah. Yeah. That kind of thing. And that’s why we do what my business does is called retrieval augmented generation. It’s like when it was GPT. Context window was 4000 tokens which a token is just like a piece of text.

Like the average word is like four tokens, I think. If you think about that and so you can imagine, okay, you can only post. Roughly around 1000 words or whatever into there as part of your question and also that usually includes the response to. So there’s a input context window and an output one, how much text they can generate.

So that kind of limits you, like how much you can put in, especially when you’re trying to answer questions from your documentation. I just crawled your website and there is what was it like almost 2000 URLs, and all the content from that that’s a lot, right? You couldn’t paste all that into chat GPT to ask questions about it, right?

Chris Badgett: But you can with DocSpot.

Aaron Edwards: ai. Yes, exactly. Because we pre process all that into getting rid of all the extra stuff and just keeping like the important chunks of text and trying to keep the chunks semantically relevant to where you’re not cutting off information between them. And then when a user asks a question, we actually search to find the most relevant pieces.

And that’s what is put in the context window for the model. But there has been. There’s been a lot of doubts if that’s like the best technique as these context windows getting bigger. So now you have GPT 4. 0, the context window is 128, 000 tokens. So that’s a big jump from 4, 000, right? The problem is you got to pay for all those tokens, right?

Cause the cost is like per token, and it’s slightly less for the input versus the output, but still that adds up a lot. So you don’t want every time you ask it or a user asks a question. To have to paste in that whole thing and pay for all that each time, you know, and there are techniques around that, like opening.

I just last week announced prompt caching. And I know Google has that and Anthropic has that now too. They’re a little bit harder to use, but basically if you’re asking the same question or at least the beginning of it. Over and over multiple times in five to 10 minutes, then it actually caches a big chunk of that’s reused each time.

And by doing that, I think that you save 50 percent on those tokens. So there are techniques around that, but still it’s inefficient to paste everything in there, but it is really useful for some things. Like I love using a Gemini. They have a 2 million in context window, which is huge. You can paste the whole book in there, right?

and do Q& A or summarization or whatever over the entire book without having to do what we do, like the reg. And the same thing like with Gemini, you can do like a video Or a bunch of images all at once because their window is so big. It’s expensive, but you can do like certain tasks that you couldn’t do before.

So we’ll see these context windows keep getting bigger and prices drop. And they introduce things like caching. I think these like chatbots and things like this for our use case are going to get even more accurate because we’ll be able to pass. Passing even more information.

Chris Badgett: And how about chain of thought reasoning, or is that a bigger advance from autocomplete or how do you, how would you describe it?

Cause obviously we want like the AI to be as good or smarter than,

Aaron Edwards: basically, like some people do

Tells it to think carefully, and how to think then it actually somehow does that internally before it gives an answer. So it’ll be like something like, all right, divide the problem into 10 steps for how you’ll solve it. And then think about if those are good steps for solving this particular problem.

And then as you generate the output, compare it to these like rules to see if that’s a good output, and it’s not like necessarily doing that internally. It’s just Telling it how to think makes it think better,

which

Aaron Edwards: is weird, right? It’s like those hacks like where people just start, Oh, if you say I’m going to get fired, if you don’t give a good answer, then it gives a better answer, these weird little quirks in the system that people like didn’t realize it’s a little bit like that.

So that’s what the chain of thought is you provide like a specific like prompt template like that, and then it. Can do a better job of more advanced like reasoning answers. And then what open AI did is they just released a new model called Oh one, if you’ve heard of that, and basically what they did is they baked in that chain of thought into the model.

So instead of having to prompt that each time, the model has actually been fine tuned with a whole bunch of examples of chain of thought. I don’t know, like really. Really advanced chain of thought prompt and then how and then the output of how it would develop that and then what the final output shown to the user should be and then they passed in whatever 100, 000 examples of that they had generated or compiled.

And use that to fine tune the model. So now it does that chain of thought internally, but on its own. So that gives you much more accurate, like things, especially if you’re wanting to do things like math or coding or it’s great if you’re trying to do something like, okay, this is. Here’s my guidelines of for how this entire like proposal should be laid out.

The outline and all the things and the logic and everything. Now here’s, I pasted in all the information, now I want you to compile that into this beautiful report. That’s 10 pages long, right? So instead of trying to just one shot that using that, Oh, one model, it’s actually does all that thinking behind the scenes and that means it’s slow because it’ll literally say, it’s like thinking about this.

Considering this, whatever, it can take five minutes sometimes for really advanced job you give it to you for it to do all that internal thinking before it spits out the final product. So it’s really cool, but it’s not like great for like customer support or chatbots to where you, you want to answer quickly.

But it is very good for things that require a lot of like complexity and thinking and reasoning to generate a good output.

Chris Badgett: And that five minutes may be like a year of human time, right? Let’s talk about something really to, for the course creator folks.

Aaron Edwards: Yeah,

Chris Badgett: there’s a lot, there’s courses and content, but there’s also coaching.

So let’s say somebody has been YouTubing on a given topic for years, or they’ve been podcasting on it for years or blogging about it, or they have a course library on their website. How could they use dark spot AI to. Answer questions, but also in more of a coaching context, so essentially a subject matter expert could turn their body of work into a like a digital mentor or coach.

Aaron Edwards: Yeah, that’s actually a great idea. I really have been wanting to play around with use cases for LMS. Cause as you mentioned, I think that is really like cool way of doing it, different than just customer support or whatever, being able to chat with, or have that kind of conversation with the course content.

So there’s the one thing to where obviously it’d be like user initiated. It’d be really fun to play around with ways to have the chat bot initiate things. It’s not a

Chris Badgett: process of what’s your biggest challenge and

Aaron Edwards: right. Yeah. That kind of thing. And perform more like reasoning things, even generating like quizzes on the fly or something like that.

Would be cool.

Chris Badgett: Or even prescribing sometimes it’s a problem. It becomes a problem when you actually have too much content and then a student or a client comes in and they’re overwhelmed. So one of the things is just like helping people find the stuff that’s most related to their biggest.

Your challenge.

Aaron Edwards: Yeah. And that we do very. Efficiently, as it is. But one of the cool things about DocSpot is you can embed it anywhere, we provide like a simple like chat widget that you can stick on your website, but we have a full featured API. So we have people like we have WordPress plugin authors that have it like built into the dashboard of the plugin.

That’s pretty cool. To where, yeah. So to where users can. Quickly, get a answer to a question about how to use the plugin and you can even pass in like custom metadata. So the chat bot can know, okay, what page it’s on, like what settings page or different things like that. Or you can pass in just random metadata, like, the, obviously the user’s name.

It can personalize responses or maybe like what plan they’re on, that kind of stuff, you know, the links to get to certain pages that are like to manage their account or manage their orders and WooCommerce or whatever it may be. You can pass all that in as metadata, as context to the chat, so that the bot is able to answer user questions, not just from your documentation, but from personalized metadata about the user and their state in your application.

So something like that would be really useful, like for Lifter LMS or whatever In courses. So it can know section of the course it’s on, like maybe what they’ve completed, different things like that, as well as being able to actually chat with the video content or chat with the course content and that kind of things.

Yeah.

Chris Badgett: This concept of a AI tutor is a big deal. Khan Academy, that guy, I know he’s done some stuff with his body of work, but in the LMS space, we’re all chasing that AI tutor, which is. Ideally would be, could be proactive, not just reactive, like we talked about with coaching versus retrieval or reacting.

Yeah. We can’t do an AI, or sorry, go ahead. Did you have something else?

Aaron Edwards: No.

Chris Badgett: We can’t do an AI podcast without addressing like fear or concern with AI. Like, how do you think about the future? Are you worried about AI taking over? For me, it seems what happens when we, when AI becomes more intelligent than the smartest human, I would argue maybe that’s already happened.

Like, how do you think about the future, particularly with the rate of change?

Aaron Edwards: I’m not that old, but I feel like I’ve lived through at least three or four major technology waves where people said, this is going to destroy society or whatever it may be, take all our jobs, and I think the pattern has always been the same, like with new technology, new tools, it just makes us more efficient.

Obviously there’s, but just like any other tool, whether it’s technology, whether it’s a gun, whether it’s money, it just like accelerates people’s base tendencies, like for good or bad, things who use for good or bad For AI, obviously like the fear is, Oh, it’s going to like, obviously the fear is like replacing jobs.

And I’m torn on this because literally my business right now is like to help replace jobs, you have a support bot that can handle like 70 percent of your tickets granted, it’s probably like tier one, so those are the people that are going to get eliminated.

First, the ones with the lowest like capability or education or that kind of things if that happens. But also it makes lives a lot. Easier for the other workers, because they don’t have to answer those boring, repetitive questions or tasks or things like that.

Chris Badgett: And if you’re using it for pre sales, like you might actually start selling more product, which is going to require more support because people don’t have to wait to get an answer about.

Will this work for me? And that’s true.

Aaron Edwards: Yeah. Or other marketing use cases. Like I use my custom trained chat bot for when I get like a lead, like someone signs up, like gives them their email or whatever on the site. I actually have the chat bot go and read their website, their business website.

Oh,

Aaron Edwards: and then use that, pass it to, My train chat bot and say, okay, I want you to draft this like really bespoke custom email. That’s recommending all the specific ways my product can be used for their business. So it writes like a totally personally, way better than I ever could with an hour of research, right?

That’s giving them specific things like, Oh, here’s how you can use a chat bot for your carwash network,

Chris Badgett: yeah. Super contextual.

Aaron Edwards: Yeah. So it allows like mass personalization. So I think it’s going to lock more things. I think just like other technology, it’s, I think for the most part is just going to be making users more efficient, making them able to achieve more.

I think obviously it will replace some jobs, but it creates new ones. Just like we’ve seen with every new technology. So I think instead of spending your time, like being scared, I think it’s a lot better to spend your time learning. Yeah. Playing the playground. Yeah. The way you succeed in this world of ours, and this has been true for a long time is learning how to learn.

So that means staying on top of whatever the latest technology or developments are playing with that, figuring out how to leverage that and use it in your workflows and your business, um, in your career, there’s that classic quote that your job might not be replaced by AI, but it’ll be replaced by someone who knows how to use AI.

Yeah,

Aaron Edwards: and I love that quote because I think it’s very true, at least right now. Maybe someday we get to that point where, like the AI takes over everything and no one has to work anymore or something. I don’t know, I don’t really see that happening. And I think the same pattern is going to replay itself.

Like things get more efficient, new jobs that no one ever imagined, are created. So I’m a bit of an accelerationist in that. Absolutely.

Chris Badgett: Can you final thing prepare that demo where people can talk to the AI? And as Aaron’s getting that ready think about this idea that somebody could talk to your website just using voice potentially in languages.

Aaron Edwards: Yeah, so this is very This is like my weekend project that I’ve been hacking together. But if you, I don’t know if you knew, but open AI just released their advanced voice mode, like both in chat, GPD plus, where you can play around with it. But last week they released an API for it. So you can actually have the real time, like conversations with AI.

And what makes this different than before is before you would have to you asked a question, it would have to first run it through a model to turn that audio into text, right? Then it would pass that text to chat, GBT, which would provide an answer in text. And then it would have to turn that text back into voice again.

latency there. Because you’re doing like three different conversions. And so like a lot of big gaps and things like that to make it just really not feel like a phone call or not feel like a real person. So their new real time API, which they just released. I was just playing with connecting that to DocSpot cause I want to create like voice agents so you can connect your phone number to it, or you can add a little widget like this that I have right here on your website.

To talk like to a human representative, human sounding representative. Obviously there’s a lot of UX things to solve. Right now there’s like a bit of a pause while it’s looking at data to answer a question, which I need to fill in those pauses, but I’ll give you a quick demo.

Hola, como puedo ayudarte hoy?

Aaron Edwards: Can you speak in

English? Absolutely. I can speak English. How can I assist you today? What kinds of questions can you answer? I can help with questions related to DocsBot testing, from features and troubleshooting to integration. Okay,

Aaron Edwards: I want to know about the pricing plans.

How about the hobby plan? How many pages does that come with?

The hobby plan for DocsBot includes up to 1, 000 source pages. If you need more detai

Aaron Edwards: Oh, that’s great.

Chris Badgett: Are you there? I’m

Aaron Edwards: here.

Chris Badgett: Yeah, that was awesome.

Hello again. Regarding your question. Let me check if you can train your bot using YouTube videos. Give me a moment. Good news. You can indeed train your DocsBot using YouTube videos. This feature allows your chat bot to provide answers. Talk

Aaron Edwards: faster. It’s too slow.

You can definitely train your DocsBot using YouTube videos. This feature lets your chat bot. All right. Thank

Aaron Edwards: you.

Glad to help. If you have more questions, feel free to ask. So that’s

Aaron Edwards: A rough demo.

Chris Badgett: That’s awesome. Thanks for giving us a glimpse into the future. It’s amazing, Aaron, what you’ve built over at Docspot AI.

So go check that out. If you’re listening to this episode, that’s docspot. ai. And how would you say Aaron was the best way for somebody to get started? They’ve listened to this episode. They’re excited. What should they do to we have

Aaron Edwards: a free plan. You don’t even need a credit card. So you can just.

Click the big, build your chat bot right now. And then you can super simply without even paying, like you can add in like URL to your site or a few different things. There’s obviously limits on how you can train your bot with the free plan, but if you need more than that, you want to try all the features.

We do have a money back guarantee. So we’ll give you 14 days to give it a try. And if it’s not good for you We’ll just give you a full refund. But for most people, it takes like less than five minutes to have a working chat bot for customer support or internal knowledge or whatever.

So it’s pretty magical.

Chris Badgett: That’s awesome, Aaron. Thank you for coming back on the show. I really appreciate it. Thanks. We’re going to have to catch up in another year and see what’s going on here. And the way the rate, this is accelerating. It might have to be before a year.

Aaron Edwards: No problem.

Chris Badgett: Super impressed.

And The way you innovate and play in the playground of technology and provide business tools is really inspiring. So thank you for coming. Thanks for coming back on the show.

All right.

And that’s a wrap for this episode of LMSCast. Did you enjoy that episode? Tell your friends and be sure to subscribe so you don’t miss the next episode. And I’ve got a gift for you over@lifterlms.com slash gift. Go to lifterlms.com/gift. Keep learning, keep taking action, and I’ll see you in the next episode.

2023 WordPress LMS Buyer’s Guide

Exclusive Download! Stop wasting time and money researching online course and membership site tech.

WordPress LMS Buyer's Guide Download Cover Images

The post AI for Course Creators with Aaron Edwards From DocsBot AI appeared first on LMScast.

  continue reading

124 episoder

Artwork
iconDel
 

In this LMScast episode, Aaron talks with Chris Badgett about AI for course creators. He shares the journey of developing DocsBot.ai, from creating a proof-of-concept chatbot trained on WordPress documentation to building a powerful tool that enables companies to leverage their own documentation for content creation, customer service, and more.

Aaron Edwards is the creator of DocsBot.ai, a cutting-edge chatbot driven by AI that improves document management and business assistance. Aaron explains the fundamental idea of retrieval-augmented generation (RAG), which blends generative.

Image of Aaron Edwards

By giving the AI context, real-world data can help it provide precise, well-founded responses. This method tackles a prevalent issue with AI, which is that it “hallucinates” or fabricates knowledge. By giving the AI context, real-world data can help it provide precise, well-founded responses. This method tackles a prevalent issue with AI, which is that it “hallucinates” or fabricates knowledge.

Data from the actual world can assist the AI give accurate and well-founded answers by providing context. A common problem with AI is that it “hallucinates” or creates knowledge, which this approach addresses.

This podcast offers insightful information on the technological foundations of DocsBot.ai and how it can be used to many businesses, from agencies to course developers, to enhance assistance, productivity, and content reuse.

Here’s Where To Go Next…

Get the Course Creator Starter Kit to help you (or your client) create, launch, and scale a high-value online learning website.

Also visit the creators of the LMScast podcast over at LifterLMS, the world’s leading most customizable learning management system software for WordPress. Create courses, coaching programs, online schools, and more with LifterLMS.

Browse more recent episodes of the LMScast podcast here or explore the entire back catalog since 2014.

And be sure to subscribe to get new podcast episodes delivered to your inbox every week.

2023 WordPress LMS Buyer’s Guide

Exclusive Download! Stop wasting time and money researching online course and membership site tech.

WordPress LMS Buyer's Guide Download Cover Images

Episode Transcript

Chris Badgett: You’ve come to the right place if you’re looking to create, launch, and scale a high value online training program. I’m your guide, Chris Badgett. I’m the co-founder of LifterLMS, the most powerful learning management system for WordPress. State of the end, I’ve got something special for you. Enjoy the show.

Hello and welcome back to another episode of LMScast. I’m joined by my friend Aaron Edwards. He’s from DocsBot. ai. That’s DocsBot. ai. Go check this out. This is not going to be your average episode where we talk about chat GVT or some basic AI concepts, although we are going to talk about that stuff some.

We’re going to go deep. Aaron has been deep in the weeds with WordPress, AI, software development, solving problems for business owners. We’re going to get into it today. This is going to be a fascinating episode, but first welcome back on the show, Aaron.

Aaron Edwards: Thank you for having me, Chris.

Chris Badgett: Good to see you again.

I always love running into you because you’re always up to like really interesting, cutting edge things. Whether it was NFT stuff in the past, solving challenges. With the media library and uploads and stuff like that. I’ve watched you over the past couple, maybe several years launched DocSpot.

So we’re going to focus on the AI component both for WordPress professionals and agencies, but also for course creators. But first, just at a high level, what is DocSpot. AI?

Aaron Edwards: Yeah, sure. The idea is let me go back. I’ll tell my story a little bit with it. As you said, I just love playing with the latest new technology.

So obviously as this generative AI stuff started coming to the forefront I was just diving in and building things, playing around with it, learning the latest Tech and stuff like that. There was images and I know we talked about in the previous episode and now doc spot and what I started out is this new technique called retrieval augmented generation.

So, chat GPT dropped in what is end of 2022, I think. And it like wowed everyone, obviously, like we’re like, Oh my gosh. But people quickly realized that what hallucinations were. So these AI models are trained to be like super, super helpful and great assistance so much so that they’ll just make stuff up to make you happy. There’s a new in that time during that fall, there’s a new technique some papers and things that have come out where People had realized that, okay, if you provide like chat GPT with context, for example, you paste in a page of text or whatever about your product or something like that you paste that in there and then you ask a question about that content that you pasted in.

Now it has like facts to ground its answers. So that way it can actually like. Provide like a accurate and valid answer by summarizing or extracting information from that text that you pasted in this context. So that’s what. This new technique called retrieval augmented generation is.

And so, I just was playing with that and I thought this would be really cool just as a proof of concept. And to learn myself is to build a chat bot that was trained on all the WordPress documentation. So I went and I like built like a web crawler that went and crawled all the documentation pages on wordpress.

org. Whether it’s the developer side or like the user kind of side, all that kind of stuff. And then. into a, it’s called a vector database, which is just a way to store that training data using AI embeddings it’s called, which is a way to do semantic search so that if you search for you give a question and it’s able to identify texts that may not have the exact same words in the question, but I do.

but able to identify pieces of text that are the same like semantic meaning or close semantic meaning. So basically that way you can transform a user’s question into an actual whatever the most relevant document pieces of documentation are to answer that question. And so I launched that and it was called, um, chat WP and it’s still online.

It’s a, let me see if I remember the URL even as WP docs dot chat. And that’s still like a free tool that’s out there. Ask me anything about WordPress. And so the kind of ideal of that was, actually I’ll share my screen. Is that all right?

Chris Badgett: Yeah, and if you’re listening to this on the podcast, just head on over to the Lifter LMS YouTube channel and look for Aaron Edwards, and you’ll find what we’re looking at here.

Aaron Edwards: Yeah, so wpdocs. chat. Just a simple chat interface, where you could ask like questions about WordPress and then it goes. And then from that training data, it’s able to provide like. Output and examples and things like that, even code snippets and stuff that is learned from the documentation.

So that was my my learning experience and proof of concepts. And then what I did is people like were super excited about it. So I slapped a little a waitlist signup form at the bottom of the page. And I said, Hey, if you’re interested in this for your business. Let me know. And I just put a little survey there what are like the key features that you need, that kind of thing.

If you’re trying to use this for your own business and I got a lot of like good responses on that, like a good wait list and the thought, all right, this is like somewhat proven that there’s some desire, for businesses to have something like this. And I spent a month working nights all night, to put out duck spot, so that was the product, which has been going well.

It’s grown a lot, it’s my full time job now. It’s been More than a year and a half. And basically what we do is we make it super easy for businesses to train their support. It can be used internally, like for your team to be able to get answers quickly from your existing documentation, files SOPs, that kind of thing it can be used.

Some of our customers use it. For, the classic content generation or repurposing. So they have their chatbot trained with all their marketing content or whatever. And they can use that to generate, ad copy or different things like that. Or if their agency, they might have a chatbot trained for each of their clients.

And they’ll use that for content repurposing a lot of different use cases.

Chris Badgett: So what was the surprise to you as people started using it? Like who was like the main group and then what were some surprising use cases that started popping up?

Aaron Edwards: Yeah. I think the biggest thing, and this is really what led it to it being a success is somehow some Japanese Twitter tech influencer.

Like from my chat bot, like in Japanese and just shared a video of it, answering questions in Japanese. And that went totally viral in Japan. It was so crazy. I like woke up in the morning and suddenly. I had 100 support tickets in Japanese and my servers are crashing. And I was getting all these signups and things like that.

I am really stressed testing it. So it’s like, What is going on? Yeah, so it went viral in Japan, which is pretty crazy. So still probably maybe a third of my customers are Japanese. And that really led me down the path of doing the work that kind of sets us apart from a lot of our competitors and that making sure that it’s very like multilingual and compatible with non English languages.

Most of all the AI announcements and benchmarks and all that stuff that you see. online and all the hype. It’s all English. Like you don’t actually realize it that, that, okay. Yeah. This tiny model that’s like amazing that like Lama or whatever that Lama three or whatever, that meta put out it sounds like so amazing, but you realize, Oh, it’s only trained on The four most common like Western languages, and the benchmarks suck on other languages.

So that’s been a really big thing, like from prompts optimization on the backend to making sure it like can detect languages properly and answer and whatever the language that the user asked questions, that kind of thing. It’s taken a lot of like work and even ongoing as I add new features, making sure that It works for all those use cases,

Chris Badgett: tell us more about like the training data or the source pages.

So for example, if I’m an agency, I may have a bunch of case studies and service offerings and a blog, bunch of blog posts about how I approach my agency work or my clients, or podcaster and I have a lot of podcast episodes. Or I’m a, I have a product that has support library of content. Like, how do people think about source pages and what could be sources?

And at what point, how much source content do you need for this to get really valuable?

Aaron Edwards: Yeah. That’s like one of the harder things that we do is like the actual like code to create these like chatbots and do the retrieval augmented generation, and that’s all open source. There’s all kinds of.

Like things that you can do and probably spend that up. If you’re a developer and just a few hours, something that kind of working as a proof of concept, but really the hard part is like figuring out how to take like real world data that businesses have. Whether it’s course content, as you mentioned, documentation, crawling a website, all that kind of thing and transforming that into something that the bot understands and is able to use to answer questions accurately, without hallucinations.

So we developed a lot of different,

Source pages or ways to, Import information to your bot. So whether it’s like uploading document files PDFs or docs, PowerPoint, Excel, whatever it is, each of those things, we have to have different logic for how we parse that and how we divide it up into pieces.

That that the LLM, the large language model AI can understand. So there’s a lot of work in that. You can train it just with like FAQs that are simple to answer. You can have a crawl your entire website, like sitemap, that kind of thing. And connected to tons of cloud sources, to your notion, confluence, Salesforce, get book places where you already have documentation stored, or if you’re using like Zen desk or help scout or intercom fresh desk, like a lot of these popular, like support tools that maybe you have your knowledge base in, or you have your previous Customer support, history tickets in there, and you can use all that to train your bot so that it can answer questions into the future.

So we’re always adding like new data sources. A fun one that I worked on last month is YouTube. Yeah, it sounds awesome. You’re mentioning that you have all these YouTube videos, right? , so being able to train your bot just by dropping in a YouTube video, URL or like a URL to a playlist of a.

Of, up to a hundred different videos. And then it goes through and actually had to, this is the hard part. YouTube does not like scraping. So it’s literally like people don’t realize that’s the hard part. I had to use like a whole network of residential proxies, so it’s actually going through like people’s home computer to scrape the subtitles from YouTube and then use that to train the bot so that you can chat with your YouTube videos.

And and even as you’re chatting with them, it like provides source links to where it got the answer. So it’ll actually link to the correct second in the video where it pulled that answer from which is fun.

Chris Badgett: Very cool. In terms of humans. Potentially being concerned about I don’t want this AI to speak for me, like how what are the options in terms of just cut it loose and let it talk to anybody versus a human reviews before it gets communicated?

Like, how do you think about that moderation aspect?

Aaron Edwards: Yeah, I think and a lot of our customers. They’re using it for frontline customer support. So tier one. So if you think about it, first of all, if you have a business, that customers never read the documentation, right? A

Chris Badgett: lot of our support is here’s the link to the doc.

Exactly.

Aaron Edwards: Yes. That’s whatever, 75 or something percent of the average company support is like here’s the link to the document where they answer that question for you, and in the, Previously, there’d be things like search tools and stuff try to identify that to try to prevent tickets being created, deflection, we call it but why not have the same thing that Google does now when you Google something and it gives an AI answer by looking at the top results and summarizing that into a succinct answer instead of having to go to all these pages and read and figure out how to answer your question.

That’s what we do. You take all the existing information that you’ve already written at some point, that’s already been recorded at some point in your documentation or support history or whatever, or YouTube demo videos, whatever it may be. And provide like an instant, like quick answer to them.

That’s grounded in that truth of your training data and also provides links to where it got that information. So they could click out and. And verify that if they want to or whatever. So yeah, that’s one side of it. And I think that these models are definitely well proven to be good enough for that kind of tier one support.

And I’m surprised like what it can answer too. So even like my bot is trained on all my like developer documentation for APIs and things like that. So I find myself like asking questions about how do I give me an example code for how I integrate my widget with Zendesk, And it literally both reads like my documentation, but it’s actually like performing logic and reasoning as well after reading that, just like a human would to generate new answers.

It’s not just copying, copying and pasting verbatim which is pretty amazing that these AI models are capable of doing that reasoning. So that even gets you more than what you’d say is the tier one support, because it’s not just. Not just linking to documentation, but it’s actually able to do some reasoning to

Chris Badgett: that’s very cool.

I know one of the things that I think about when, looking at AI tools is cost and we don’t have to get into the specifics of your pricing because I know it might change. But As the better AI gets, it might require more resources, which could in theory, make it more expensive. Or you need a large training set.

Like, how do you think about pricing and making it work for everybody? The the business owner, you making sure it’s a profitable venture. Like, how do you think about that?

Aaron Edwards: Man, when I launched the same week that the. Chat GPT API became available which was great timing, right? But the costs for that, like even the old, like 3.

5 model, chat, GPT model it would cost, I think three or four cents per question per user question. Now, if you’re getting like 10, 000 questions a month or

something that adds up fast, right? But at the same time, if you compare that to hiring a support person, the answer, those Whatever, 70 percent of those 10, 000 questions. That’s still super cheap, right? It was hard thing to, to factor in. And so what I actually did in the beginning is we did bring your own API key.

So you could actually put in your key from open AI, like into our dashboard and we’d save it like encrypted and use that At cost. So instead of reselling credits or something like that, like some of our competitors do with a markup, we just basically at cost, for the actual processing.

And what’s amazing is in, in less than two years, the The price for how much power you get, like from the AI model has literally dropped more than a hundred times. It’s yeah, it’s been crazy. And as a business who’s like trying to figure out like my pricing models and things like that, and how to charge.

That’s been great both for my customers who are like bringing their own key, but now I’m able to provide access like included to make it simpler for people to get started. They don’t have to have their own key, so like I heard, I think it was

said that if you’re building products on AI, like it’s changing so fast and its capabilities are changing so fast, you need to be building like on the cutting edge to where right now. It’s seems super expensive or super like impossible for it to do. Because it’s just change every two to three months.

It’s like a new model comes out two to three times price drop. It’s amazing.

Chris Badgett: Help us understand like the models in the background are you leveraging open AI or you’re leveraging llama, a combination of stuff? Like how does it, like what powers all this?

Aaron Edwards: Yeah, right now we’re using only a open AI models just cause they have the best, like developer ecosystem and things like that.

We are playing with other models like Gemini and Llama. Maybe, like I said, there’s still weaknesses there for other languages. So that’s one of the things that we have to test heavily is we want to support having a chat bot in Japanese, right? And that it’s going to be able to, no matter what language the user asks a question, that it’s able to still find the right documentation, even though maybe your documentation is in English.

If someone asks a question in Japanese, it should be able to use AI to find the relevant documentation and provide an accurate answer to the user. And so for that, you need some of these larger models that are, have much more training data from other languages than English. So right now that’s the GPT 4.

0. That’s what kind of like our workhorses. So food like for free, like along with your plan. And that works for almost all use cases and it’s super fast and it’s super good. Multilingual as well. And then if you like, you want Much more advanced reasoning, like to where it can maybe do some custom coding and things like that as part of its answers.

Then you could jump up to the full, like GPT 4. 0 model, which has dropped down in price substantially to, it still costs less than one cent per question, more like 0. 5 really. So super cost effective. And they just keep getting more and more powerful.

Chris Badgett: Could you describe to the layman, just to highlight the rate of change with AI what’s the difference with 4.

0 in OpenAI versus 3. 5 or Turbo or whatever came before?

Aaron Edwards: Yeah a lot of it’s like proprietary, so they don’t tell us like all the technical details you can infer. There’s been a lot of new inventions. So first of all, was just like the amount of data is trained on, right? Like call that parameters, like how many parameters, that’s the size of the model.

And so like you had Like GPT two, I think. And then GPT three which is like the first ones that they had like an API. If you were really lucky, you could get a key to use that one. And that’s what some of the early like content creation things, um, used and then 3. 5 was basically, they took that same model and they did what’s called RLHF, which is basically they they Took had humans, right?

A bunch of okay, here’s the message. And then here’s what a good response is for that. And so they hired, armies of people and in Africa and other third world countries that speak English and other data set online to do that reinforcement learning on top of the existing model, which is basically where the invention of chat GPT came from.

So instead of being something where you had to figure out these weird Ways of prompting it crazily to where it like completes, like what you started. Now you can just give any kind of instruction and it just knows how to follow that instruction and provide a good output. So that didn’t really change the size of the model.

It just made it much more useful to the average user. And then you had the next big jump in model size and that was GPD four. And they say that went from GPD three is probably something like four or 500 Billion parameters. I think they estimate maybe three or 400 billion parameters.

And GPD four is probably like closer to 2 trillion. Wow. Yeah. And that’s after being pruned and cleaned up and that kind of stuff. It’s more like a tenfold increase, in size. And so that’s why it’s so much more expensive and slower. Cause it’s got to run on a whole cluster of 20, 000 GPUs, because it uses so much memory.

To do all that kind of stuff. And then I’ll put on top of that, then they’re able to do what’s called distillation, which is where you take that big model and you pull out by using like the reinforcement learning and things like that training, what’s the most useful. Parts to you, so it may not have the widespread knowledge from all that training that pulled in, but all the things that people most commonly ask, or most commonly use it for logic, it kept only those parts so you can have a 10 times smaller model that was trained that was distilled from the big one.

I’m just pulling out the most useless things. And that’s what GPT 40 mini is which is like their most cost effective model. It’s super, super fast, super inexpensive, but still very powerful, even in other languages.

Chris Badgett: That’s awesome. And Aaron, while I’ve got you an AI expert and who’s great at teaching and communicating this stuff.

Help me understand the buzz around two concepts. I’m not sure if they’re related or not, but one is having a large context window and the other is chain of thought reasoning. What are those?

Aaron Edwards: Okay. They’re not like necessarily related, but the context window is basically like How much text or video or whatever the input, how much the AI model can handle at once.

Chris Badgett: If you’re using chat, GBT, sometimes you’re like, you try to copy paste and it’s too long or whatever.

Aaron Edwards: Yeah. Yeah. That kind of thing. And that’s why we do what my business does is called retrieval augmented generation. It’s like when it was GPT. Context window was 4000 tokens which a token is just like a piece of text.

Like the average word is like four tokens, I think. If you think about that and so you can imagine, okay, you can only post. Roughly around 1000 words or whatever into there as part of your question and also that usually includes the response to. So there’s a input context window and an output one, how much text they can generate.

So that kind of limits you, like how much you can put in, especially when you’re trying to answer questions from your documentation. I just crawled your website and there is what was it like almost 2000 URLs, and all the content from that that’s a lot, right? You couldn’t paste all that into chat GPT to ask questions about it, right?

Chris Badgett: But you can with DocSpot.

Aaron Edwards: ai. Yes, exactly. Because we pre process all that into getting rid of all the extra stuff and just keeping like the important chunks of text and trying to keep the chunks semantically relevant to where you’re not cutting off information between them. And then when a user asks a question, we actually search to find the most relevant pieces.

And that’s what is put in the context window for the model. But there has been. There’s been a lot of doubts if that’s like the best technique as these context windows getting bigger. So now you have GPT 4. 0, the context window is 128, 000 tokens. So that’s a big jump from 4, 000, right? The problem is you got to pay for all those tokens, right?

Cause the cost is like per token, and it’s slightly less for the input versus the output, but still that adds up a lot. So you don’t want every time you ask it or a user asks a question. To have to paste in that whole thing and pay for all that each time, you know, and there are techniques around that, like opening.

I just last week announced prompt caching. And I know Google has that and Anthropic has that now too. They’re a little bit harder to use, but basically if you’re asking the same question or at least the beginning of it. Over and over multiple times in five to 10 minutes, then it actually caches a big chunk of that’s reused each time.

And by doing that, I think that you save 50 percent on those tokens. So there are techniques around that, but still it’s inefficient to paste everything in there, but it is really useful for some things. Like I love using a Gemini. They have a 2 million in context window, which is huge. You can paste the whole book in there, right?

and do Q& A or summarization or whatever over the entire book without having to do what we do, like the reg. And the same thing like with Gemini, you can do like a video Or a bunch of images all at once because their window is so big. It’s expensive, but you can do like certain tasks that you couldn’t do before.

So we’ll see these context windows keep getting bigger and prices drop. And they introduce things like caching. I think these like chatbots and things like this for our use case are going to get even more accurate because we’ll be able to pass. Passing even more information.

Chris Badgett: And how about chain of thought reasoning, or is that a bigger advance from autocomplete or how do you, how would you describe it?

Cause obviously we want like the AI to be as good or smarter than,

Aaron Edwards: basically, like some people do

Tells it to think carefully, and how to think then it actually somehow does that internally before it gives an answer. So it’ll be like something like, all right, divide the problem into 10 steps for how you’ll solve it. And then think about if those are good steps for solving this particular problem.

And then as you generate the output, compare it to these like rules to see if that’s a good output, and it’s not like necessarily doing that internally. It’s just Telling it how to think makes it think better,

which

Aaron Edwards: is weird, right? It’s like those hacks like where people just start, Oh, if you say I’m going to get fired, if you don’t give a good answer, then it gives a better answer, these weird little quirks in the system that people like didn’t realize it’s a little bit like that.

So that’s what the chain of thought is you provide like a specific like prompt template like that, and then it. Can do a better job of more advanced like reasoning answers. And then what open AI did is they just released a new model called Oh one, if you’ve heard of that, and basically what they did is they baked in that chain of thought into the model.

So instead of having to prompt that each time, the model has actually been fine tuned with a whole bunch of examples of chain of thought. I don’t know, like really. Really advanced chain of thought prompt and then how and then the output of how it would develop that and then what the final output shown to the user should be and then they passed in whatever 100, 000 examples of that they had generated or compiled.

And use that to fine tune the model. So now it does that chain of thought internally, but on its own. So that gives you much more accurate, like things, especially if you’re wanting to do things like math or coding or it’s great if you’re trying to do something like, okay, this is. Here’s my guidelines of for how this entire like proposal should be laid out.

The outline and all the things and the logic and everything. Now here’s, I pasted in all the information, now I want you to compile that into this beautiful report. That’s 10 pages long, right? So instead of trying to just one shot that using that, Oh, one model, it’s actually does all that thinking behind the scenes and that means it’s slow because it’ll literally say, it’s like thinking about this.

Considering this, whatever, it can take five minutes sometimes for really advanced job you give it to you for it to do all that internal thinking before it spits out the final product. So it’s really cool, but it’s not like great for like customer support or chatbots to where you, you want to answer quickly.

But it is very good for things that require a lot of like complexity and thinking and reasoning to generate a good output.

Chris Badgett: And that five minutes may be like a year of human time, right? Let’s talk about something really to, for the course creator folks.

Aaron Edwards: Yeah,

Chris Badgett: there’s a lot, there’s courses and content, but there’s also coaching.

So let’s say somebody has been YouTubing on a given topic for years, or they’ve been podcasting on it for years or blogging about it, or they have a course library on their website. How could they use dark spot AI to. Answer questions, but also in more of a coaching context, so essentially a subject matter expert could turn their body of work into a like a digital mentor or coach.

Aaron Edwards: Yeah, that’s actually a great idea. I really have been wanting to play around with use cases for LMS. Cause as you mentioned, I think that is really like cool way of doing it, different than just customer support or whatever, being able to chat with, or have that kind of conversation with the course content.

So there’s the one thing to where obviously it’d be like user initiated. It’d be really fun to play around with ways to have the chat bot initiate things. It’s not a

Chris Badgett: process of what’s your biggest challenge and

Aaron Edwards: right. Yeah. That kind of thing. And perform more like reasoning things, even generating like quizzes on the fly or something like that.

Would be cool.

Chris Badgett: Or even prescribing sometimes it’s a problem. It becomes a problem when you actually have too much content and then a student or a client comes in and they’re overwhelmed. So one of the things is just like helping people find the stuff that’s most related to their biggest.

Your challenge.

Aaron Edwards: Yeah. And that we do very. Efficiently, as it is. But one of the cool things about DocSpot is you can embed it anywhere, we provide like a simple like chat widget that you can stick on your website, but we have a full featured API. So we have people like we have WordPress plugin authors that have it like built into the dashboard of the plugin.

That’s pretty cool. To where, yeah. So to where users can. Quickly, get a answer to a question about how to use the plugin and you can even pass in like custom metadata. So the chat bot can know, okay, what page it’s on, like what settings page or different things like that. Or you can pass in just random metadata, like, the, obviously the user’s name.

It can personalize responses or maybe like what plan they’re on, that kind of stuff, you know, the links to get to certain pages that are like to manage their account or manage their orders and WooCommerce or whatever it may be. You can pass all that in as metadata, as context to the chat, so that the bot is able to answer user questions, not just from your documentation, but from personalized metadata about the user and their state in your application.

So something like that would be really useful, like for Lifter LMS or whatever In courses. So it can know section of the course it’s on, like maybe what they’ve completed, different things like that, as well as being able to actually chat with the video content or chat with the course content and that kind of things.

Yeah.

Chris Badgett: This concept of a AI tutor is a big deal. Khan Academy, that guy, I know he’s done some stuff with his body of work, but in the LMS space, we’re all chasing that AI tutor, which is. Ideally would be, could be proactive, not just reactive, like we talked about with coaching versus retrieval or reacting.

Yeah. We can’t do an AI, or sorry, go ahead. Did you have something else?

Aaron Edwards: No.

Chris Badgett: We can’t do an AI podcast without addressing like fear or concern with AI. Like, how do you think about the future? Are you worried about AI taking over? For me, it seems what happens when we, when AI becomes more intelligent than the smartest human, I would argue maybe that’s already happened.

Like, how do you think about the future, particularly with the rate of change?

Aaron Edwards: I’m not that old, but I feel like I’ve lived through at least three or four major technology waves where people said, this is going to destroy society or whatever it may be, take all our jobs, and I think the pattern has always been the same, like with new technology, new tools, it just makes us more efficient.

Obviously there’s, but just like any other tool, whether it’s technology, whether it’s a gun, whether it’s money, it just like accelerates people’s base tendencies, like for good or bad, things who use for good or bad For AI, obviously like the fear is, Oh, it’s going to like, obviously the fear is like replacing jobs.

And I’m torn on this because literally my business right now is like to help replace jobs, you have a support bot that can handle like 70 percent of your tickets granted, it’s probably like tier one, so those are the people that are going to get eliminated.

First, the ones with the lowest like capability or education or that kind of things if that happens. But also it makes lives a lot. Easier for the other workers, because they don’t have to answer those boring, repetitive questions or tasks or things like that.

Chris Badgett: And if you’re using it for pre sales, like you might actually start selling more product, which is going to require more support because people don’t have to wait to get an answer about.

Will this work for me? And that’s true.

Aaron Edwards: Yeah. Or other marketing use cases. Like I use my custom trained chat bot for when I get like a lead, like someone signs up, like gives them their email or whatever on the site. I actually have the chat bot go and read their website, their business website.

Oh,

Aaron Edwards: and then use that, pass it to, My train chat bot and say, okay, I want you to draft this like really bespoke custom email. That’s recommending all the specific ways my product can be used for their business. So it writes like a totally personally, way better than I ever could with an hour of research, right?

That’s giving them specific things like, Oh, here’s how you can use a chat bot for your carwash network,

Chris Badgett: yeah. Super contextual.

Aaron Edwards: Yeah. So it allows like mass personalization. So I think it’s going to lock more things. I think just like other technology, it’s, I think for the most part is just going to be making users more efficient, making them able to achieve more.

I think obviously it will replace some jobs, but it creates new ones. Just like we’ve seen with every new technology. So I think instead of spending your time, like being scared, I think it’s a lot better to spend your time learning. Yeah. Playing the playground. Yeah. The way you succeed in this world of ours, and this has been true for a long time is learning how to learn.

So that means staying on top of whatever the latest technology or developments are playing with that, figuring out how to leverage that and use it in your workflows and your business, um, in your career, there’s that classic quote that your job might not be replaced by AI, but it’ll be replaced by someone who knows how to use AI.

Yeah,

Aaron Edwards: and I love that quote because I think it’s very true, at least right now. Maybe someday we get to that point where, like the AI takes over everything and no one has to work anymore or something. I don’t know, I don’t really see that happening. And I think the same pattern is going to replay itself.

Like things get more efficient, new jobs that no one ever imagined, are created. So I’m a bit of an accelerationist in that. Absolutely.

Chris Badgett: Can you final thing prepare that demo where people can talk to the AI? And as Aaron’s getting that ready think about this idea that somebody could talk to your website just using voice potentially in languages.

Aaron Edwards: Yeah, so this is very This is like my weekend project that I’ve been hacking together. But if you, I don’t know if you knew, but open AI just released their advanced voice mode, like both in chat, GPD plus, where you can play around with it. But last week they released an API for it. So you can actually have the real time, like conversations with AI.

And what makes this different than before is before you would have to you asked a question, it would have to first run it through a model to turn that audio into text, right? Then it would pass that text to chat, GBT, which would provide an answer in text. And then it would have to turn that text back into voice again.

latency there. Because you’re doing like three different conversions. And so like a lot of big gaps and things like that to make it just really not feel like a phone call or not feel like a real person. So their new real time API, which they just released. I was just playing with connecting that to DocSpot cause I want to create like voice agents so you can connect your phone number to it, or you can add a little widget like this that I have right here on your website.

To talk like to a human representative, human sounding representative. Obviously there’s a lot of UX things to solve. Right now there’s like a bit of a pause while it’s looking at data to answer a question, which I need to fill in those pauses, but I’ll give you a quick demo.

Hola, como puedo ayudarte hoy?

Aaron Edwards: Can you speak in

English? Absolutely. I can speak English. How can I assist you today? What kinds of questions can you answer? I can help with questions related to DocsBot testing, from features and troubleshooting to integration. Okay,

Aaron Edwards: I want to know about the pricing plans.

How about the hobby plan? How many pages does that come with?

The hobby plan for DocsBot includes up to 1, 000 source pages. If you need more detai

Aaron Edwards: Oh, that’s great.

Chris Badgett: Are you there? I’m

Aaron Edwards: here.

Chris Badgett: Yeah, that was awesome.

Hello again. Regarding your question. Let me check if you can train your bot using YouTube videos. Give me a moment. Good news. You can indeed train your DocsBot using YouTube videos. This feature allows your chat bot to provide answers. Talk

Aaron Edwards: faster. It’s too slow.

You can definitely train your DocsBot using YouTube videos. This feature lets your chat bot. All right. Thank

Aaron Edwards: you.

Glad to help. If you have more questions, feel free to ask. So that’s

Aaron Edwards: A rough demo.

Chris Badgett: That’s awesome. Thanks for giving us a glimpse into the future. It’s amazing, Aaron, what you’ve built over at Docspot AI.

So go check that out. If you’re listening to this episode, that’s docspot. ai. And how would you say Aaron was the best way for somebody to get started? They’ve listened to this episode. They’re excited. What should they do to we have

Aaron Edwards: a free plan. You don’t even need a credit card. So you can just.

Click the big, build your chat bot right now. And then you can super simply without even paying, like you can add in like URL to your site or a few different things. There’s obviously limits on how you can train your bot with the free plan, but if you need more than that, you want to try all the features.

We do have a money back guarantee. So we’ll give you 14 days to give it a try. And if it’s not good for you We’ll just give you a full refund. But for most people, it takes like less than five minutes to have a working chat bot for customer support or internal knowledge or whatever.

So it’s pretty magical.

Chris Badgett: That’s awesome, Aaron. Thank you for coming back on the show. I really appreciate it. Thanks. We’re going to have to catch up in another year and see what’s going on here. And the way the rate, this is accelerating. It might have to be before a year.

Aaron Edwards: No problem.

Chris Badgett: Super impressed.

And The way you innovate and play in the playground of technology and provide business tools is really inspiring. So thank you for coming. Thanks for coming back on the show.

All right.

And that’s a wrap for this episode of LMSCast. Did you enjoy that episode? Tell your friends and be sure to subscribe so you don’t miss the next episode. And I’ve got a gift for you over@lifterlms.com slash gift. Go to lifterlms.com/gift. Keep learning, keep taking action, and I’ll see you in the next episode.

2023 WordPress LMS Buyer’s Guide

Exclusive Download! Stop wasting time and money researching online course and membership site tech.

WordPress LMS Buyer's Guide Download Cover Images

The post AI for Course Creators with Aaron Edwards From DocsBot AI appeared first on LMScast.

  continue reading

124 episoder

Alle episoder

×
 
Loading …

Velkommen til Player FM!

Player FM is scanning the web for high-quality podcasts for you to enjoy right now. It's the best podcast app and works on Android, iPhone, and the web. Signup to sync subscriptions across devices.

 

Hurtig referencevejledning