AI Bootcamp Q&A

[00:00 - 00:09] I'll go ahead. Yeah, just some feedback. I'm a huge new line fan. I've read your books. I've taken the tiny house course and some of some others as well. Yeah.

[00:10 - 00:28] I've tracked you guys. You're one of the few emails I get that I actually read because you guys always have cool tidbits in it. I definitely see the value in this. So I'll give it some thought. But the just curious, when does it start? So it should start. So we're trying to finalize the date.

[00:29 - 01:21] It's a little bit dependent on Alvin's kind of a start date with some of his travel, but like within the next month. Anyone else have any questions about this? Yeah, it definitely sounded like the answer to my question about where do I get started with a I'm just a little concerned. What is the time commitment being that I have a full-time job? Yeah. And secondly, don't have that kind of money just to fork out right in the beginning. I mean, I'd like to do something but he says there's a payment plan for 1600 per month, but that's still gonna drag me down a little bit. So are you willing to work with me on that? Yeah, we can pick it offline too.

[01:22 - 03:01] We can discuss offline on your kind of specific and I'll provide a calendar link if you want to basically discuss the specifics with me. But in general, we're going to adapt it to your schedule. There's another kind of a book that's designed for professionals that have a full-time job. They schedule it on Sundays and then Thursdays after work. But we realize that this workshop is basically like during some people's about work time. But on a recurring basis, people may not be able to do that on a recurring basis and Alvin has his roles as well. We're going to design it so that it's it's accommodative around about people's schedules. Yeah. And then I think what will basically be helpful is to understand that we're providing a guarantee. So if you have to basically come in with something that you want to build and then the intent is basically to help you achieve that, at least basically about the prototype and whether it's the design, the architecture, or whatever it is. So I think that type of like you can almost think of this as almost built-in consulting with basically a course, not like a traditional kind of a bootcamp where you get the material and it's for junior engineers and there's not necessarily always a goal at the very end. So I've got one other question because I'm just curious. One of the areas where LLMs have blown me away recently is obviously like in the code generation side of things.

[03:02 - 04:32] I've been using Vercel's dev zero and it sometimes it makes mistakes but I've asked it to ref actor code that I know exactly how I would refactor and it does it perfectly. Is that kind of application just too big or too? If I said to you I want to come in with a project and I want to be able to use generative AI to build data pipeline tasks that can augment training data and that can do ETL processing and that kind of thing with good prompts. Do you think that's just too much or is that reasonable? I think the very beginning of the project basically, so you mentioned basically like Vercel foundational models once you basically get large require a lot of processing but if you mineralize it and you bake it down to the foundation it should basically be doable. So that's my take on it is let's assume that basically you don't build like a full-blown data pipeline it's basically it's like a toy it's like a not a toy pipeline but basically like a prototype pipeline. Basically it's not like a full ETL basically and then but all the data will basically be able to scale up as needed. I think the concepts basically remain and can basically be very easily applied so correct meaning if I'm wrong Alvin. So it sounds reasonable. I guess we can chat more about this later as well.

[04:33 - 05:49] I didn't fully catch all the details but I don't think it sounds at the very high level it doesn't sound to like it doesn't sound overly ambitious. Sounds like definitely achievable. I think what Maya wants to do is like a vertical vertical from foundational model designed for code basically. I think one thing that is very different about this whole LLM stuff in particular is the cost of experimentation and the cost of training and I've been a founder of several companies and you could say oh you raised venture capital to hire engineers so yeah it was costly to develop the apps but this is a whole new sort of level of cost and seemingly to me but you guys could comment on that it would be interesting to me. I worked just my recent experience there and this is just in speech and vision. Our GPU spend was a million dollars a year so it's definitely something that can run out of control if you're not careful.

[05:50 - 12:39] Yeah my take on it this is just seen by generative AI companies is there's multiple layers of generative AI companies so if you're talking about like horizontal generative AI companies ChachiPi it's like on the orders of like tens of millions of dollars to basically build up to a pre-training kind of a run even more if you include fine tuning but as you basically get into more vertical specific cost solutions the cost starts going down and there's also you know I'm sure all of you guys are familiar with it more is law and there's also a rapidly evolving they call it like the commoditization of the teraflop and there's a series of companies that have been working on it that have been driving the cost down per teraflop so you basically have another aspect of Moore's law that's basically happening and then there are many applications that don't require building of foundational models and then the vertical foundational models cost less like a close specific foundational model will basically cost less and some of these can be built by doing a GPU cluster at home which is a ton of electricity and kind of in some kind of a hardware time and other things but it's not always the case that you need the 100s and h 100s on Amazon and trying to basically preserve reserves this so that's like my general kind of a rough take I don't know your thoughts Allen yeah I have a lot of thoughts but yeah sounds about right you can talk more about this at the end as well yeah and and then the other aspect is you know there's effectively I mentioned previously there's multiple layers so you have four dental vertical elements or multi-mod al language models and then what you have is you have fine-tuning which is less expensive then you have retrieval augmented generation which is less expensive for example we we at Newline have a AI tutor that's hooked up into every one of these courses now and and so it uses retrieval augmented generation it's hooks up with private data it scrapes kind of additional data for getting about concepts and it's not nearly as expensive as a foundational model and so it really depends and so I mentioned basically about some of these examples partially because these examples of AI are solo founders that have limited resources that have a full-time job and the reason why I mentioned this is not the same as an indie hacker like I'm going to code a SAS application or product application on the side like these people are using AI and so they're either doing fine-tun ing or they're using a rag or they're using kind of other things so you don't necessarily have to have all the computing kind of power if you can basically do instructional fine-tuning and other kind of about aspects that are a little bit upper on the layer side I'm not sure if that makes sense I think it's good to know for would-be founders what the minimum amount they need to raise or come up with from their own assets to make a go of something yeah why is it's a little bit misleading and I think like in FinTech okay there 's plaid and a couple of other APIs that essentially charge a toll for every user you try and on board and so you could create a gorgeous app a very useful app but unless you have the money to fund the trials you're not going to get off the ground so I think there's a certain entrepreneurial thread to all this if you're trying to encourage founders if you're not and you 're just trying to equip people with the skills they need to get these jobs then you don't need that I guess what I'm trying to point out Ken is that there's a series of application-oriented AI applications that don't require the foundational model kind of about compute and and the entire idea is with the curriculum yeah but even a GPU cluster at home is not going to cost 500 dollars so it's just being realistic with founders and helping them understand what they're getting into yeah we can certainly model the cost for people but just for context we basically built that AI multiple AI applications at Newline both for operational elements and internal elements and we didn't build the internal cluster like it was fairly cost effective like less than 500 dollars 400 dollars a month basically and GPU clusters that I have a GPU cluster at home I built that use like I spent 10,000 dollars building that it certainly it's not on the grand scheme of things if you were to build a SaaS application with even one or two engineers it's going to cost you more than 10,000 dollars I think there's certainly with this there's a order of magnitude as you basically go up the order of magnitude you you are able to it exponentially cost more but basically like we have personal experience with building AI application then so so it's now like basically I'm trying to basically say hey all of this is basically is very expensive so that's great yeah did you guys have any other convolved questions about this and then I'll send you guys a calendly link so you guys can book a time with me if you want to discuss the details yeah and could you share those the details here that you've shared here in some form that'd be great just be able to read it over again yeah all right okay if you guys don't have any more questions I'll push it to Alvin that was good