Raymond Camden

Animated video backgrounds via a Web Component and ColorThief

2026-04-30T18:00:00+00:00

Earlier this year, the epic ColorThief library had a pretty significant update. I blogged about a simple demo I built with it but I was fascinated by one particular demo on their site.

The "observe" function in ColorThief lets you monitor a video source and grab the colors at a particular frame. Their demo uses this to create a lovely shadow background of the video. I believe some TVs have this feature as well, and honestly I'd worry that would get annoying, but the ColorThief demo was pretty cool, so I thought I'd try to build it with a web component.

The idea would be - take any basic video element and wrap it like so:

<video-bgshadow>
<video controls width="250">
    <source src="videos/flower.mp4" type="video/mp4">
</video>
</video-bgshadow>

The web component would then handle:

Loading the ColorThief library
Waiting for the video to be played
Running the observe method and updating the CSS

All in all, this wasn't too difficult. I don't think my shadow is as good as the demo (and I'm totally open to people submitting a PR!), but it came out ok.

I'll link to the demo below, but here's a simple example in a CodePen:

See the Pen <video-bgshadow> by Raymond Camden (@cfjedimaster) on CodePen.

Alright, so here's the code:

class VideoBGShadowComponent extends HTMLElement {
	
	constructor() {
		super();
	}
	
	async connectedCallback() {
		this.videoEl = this.querySelector('video');
		if(!this.videoEl) {
			console.warn('No <video> element found.');
			return;
		}

		// wrap the video in a new div
		this.wrapper = document.createElement('div');
		this.videoEl.parentNode.insertBefore(this.wrapper, this.videoEl);
		this.wrapper.appendChild(this.videoEl);
		this.wrapper.style.display = 'inline-block';
		this.videoEl.style.verticalAlign = 'bottom';
		if(!window.ColorThief) await this.loadCF();
		this.videoEl.addEventListener('play', this.startShadow.bind(this));
		this.videoEl.addEventListener('ended', this.endShadow.bind(this));
		this.videoEl.addEventListener('pause', this.endShadow.bind(this));

	
	}

	// Sets window.ColorThiefLoading (Promise) to deduplicate concurrent script injection across multiple instances.
	async loadCF() {
		if (!window.ColorThiefLoading) {
			window.ColorThiefLoading = new Promise((resolve) => {
				const script = document.createElement('script');
				script.type = 'text/javascript';
				script.src = 'https://unpkg.com/colorthief@3/dist/umd/color-thief.global.js';
				document.head.appendChild(script);
				script.onload = resolve;
			});
		}
		return window.ColorThiefLoading;
	}

	startShadow(e) {
		console.log('video play');
		let thatWrapper = this.wrapper;
		this.controller = ColorThief.observe(e.target, {
		    throttle: 200,
		    colorCount: 5,
			  onChange(palette) {
	            const [dominant] = palette;
                thatWrapper.style.setProperty('--glow-color', dominant.css());
                thatWrapper.style.boxShadow = '15px 15px 20px 8px var(--glow-color)';
		    },
		})
	}

	endShadow() {
		console.log('video play end');
		this.controller.stop();
	}

}

if(!customElements.get('video-bgshadow')) customElements.define('video-bgshadow', VideoBGShadowComponent);

I don't think there's anything necessarily interesting in here, although I struggled quite a bit with loadCF. I didn't want to add the ColorThief library N times to the page. Checking for window.ColorThief only works if for some reason a video wrapped with the component is added to the page after the library loads. I used Claude to help me with this bit and while it "litters" the window object with a value, I think that is a fair trade off to ensure only one library is loaded. (Technically this could be further updated to first see if ColorThief exists in general as it's possible the website uses it for something else.)

You can see a demo with a couple of examples here: https://cfjedimaster.github.io/webdemos/video-bgshadow/

And if you think this is a good start but could be so much better, I agree, help me out over at the repo: https://github.com/cfjedimaster/webdemos/tree/master/video-bgshadow

Photo by Matthew Ansley on Unsplash

Links For You (4/26/26)

2026-04-26T18:00:00+00:00

I was supposed to post this last week (I try to keep to a schedule of every two weeks), but I didn't get around to it because... nope, that's it. That's the reason. Because. And that's good enough, amiright!?!? The heat is slowly cranking up here in Louisiana and I'm dreading the full on summer, but things do slow down a bit when the kids aren't in school and that's something I greatly appreciate. Before getting into this weeks links, I was reminded a few weeks back that my wife actually reads my posts so... hi baby, I love you.

Super useful web components FTW - <form-saver>

First up is a really simple and really useful web component, form-saver. You can wrap any form with the component and instantly get client-side storage of form contents until the form is submitted. This works for all types of form fields except file fields of course. (I assume folks know this but you can't use JavaScript to set the value of a file field for security reasons.)

Here's a simple usage example from the docs:

<form-saver>
  <form action="/contact" method="post">
    <label>
      Name
      <input name="name" autocomplete="name" />
    </label>
    <label>
      Email
      <input name="email" type="email" autocomplete="email" />
    </label>
    <button type="submit">Send</button>
  </form>
</form-saver>

Isn't that sweet? Thanks go to Aaron Gustafson.

As a reminder (and I usually try to avoid linking to my own stuff in these posts, but it's definitely related), if you like this you may like my table-sorter web component as well.

Next up is a superb guide at packing Python code for distribution. I've written a lot of Python code, but have only created a distribution once or twice, and this guide literally walks you from the first line of code to publication.

Thanks to Stephen Funk for writing this up!

The Last Quiet Thing

Finally, this essay, "The Last Quiet Thing", is a thought provoking deep look at how much of our lives are being stolen from devices that constantly, endlessly need our attention. Not only is it incredibly well written, it's also really well designed as well.

This was written by Terry Godier and thanks go to Salma Alam-Naylor for sharing it on her newsletter.

Just For Fun

I love building silly things on the web (see yesterday's post as an example), and this little toy from Wes Bos is just that. Tab Snitch does one thing - set a custom title for the web page - but does so with silly and quite embarrassing titles. Although I have to be honest - a few of the fake titles listed there are one's I've probably legitimately had on my screen at some point in time. You can guess which.

Another Game: My Little Mortal Combat

2026-04-25T18:00:00+00:00

Hello awesome readers! I'm happy to announce my latest web game, My Little Mortal Combat, a mashup of two epic franchises, My Little Pony and Mortal Kombat. This began as an idea, just the name, that I recorded in Microsoft To Do in September of 2019. Yes, almost seven years ago. It sat there, at the bottom of my 'idea' list, until about a month ago when in the shower (not joking), it popped up in my head along with the basic mechanics of how the game would play.

Right now the game is just missing one feature (I'd rather not talk about until I figure out how I'm going to do it) but definitely needs some balancing work. I enjoy playing games without knowing the details of how things work, so if's that you too, head over to the game now and good luck!

https://cfjedimaster.github.io/webdemos/my_little_mortal_combat/

How I Built It

As a web app, I kept it pretty simple. Just HTML, CSS, JavaScript, and Alpine.js. I used AI (Cursor's IDE specifically) to create the UI for the three phases of the game - into, main display, and combat. I also used AI to generate some of the strings used in the game. Opponents have random "evil"-ish titles and I wrote some and then asked AI to generate some more. Some examples:

Life Eater
Hoof Smasher
the Blood Soaked
the Blood Drinker

Opponents also have random annoying facts. Again, I wrote some, had AI generate some more.

doesn't return library books on time.
likes to ruin the end of movies.
has been known to sneeze at the buffet.
steals candy from babies and then throws the candy in the trash—in front of them!
only speaks in passive-aggressive voice.
will point out your least favorite body parts.

The actual names of the opponents come directly from a My Little Pony API I found that was open source.

Here's an example of a randomly generated opponent:

Combat is basic "Rock Paper Scissors" style where you have 3 choices (Attack, Defend, Vogue) and the result is based on what your opponent does.

Your character, and the opponents, have numerical value for Attack, Defend, and Vogue. When you win a round in a fight, the damage you do is based on that skill. Your total HP is based on level.

As you can play, if you win or lose, you get gold and XP. Obviously you get a lot more when you win. You can use the gold to train skills. Your XP turns into your level which improves your HP.

As I said, I definitely think the numbers need tweaking probably, so let me know. You can check out all the code here: https://github.com/cfjedimaster/webdemos/tree/master/my_little_mortal_combat

Don't forget, I've got this game and my others listed over on my my Stuff page. Enjoy!

Photo by Felis Amafeles on Unsplash

Building a Simple Markdown PWA App

2026-04-20T18:00:00+00:00

While I didn't share it on the blog, last week I tasked Claude with using Electron to build a Markdown viewer app. It was part test (how well can Claude work with Electron) and part real need - I work with Markdown files all the time but didn't have a simple "view focused" application for it. I was sure there open source or paid app options out there, but I wanted my own. Claude did a pretty good job (you can see the source here) but one thing stood out to me - the size of the bundled app.

I created both a Mac and Windows distribution and both were around 90 megs. That's not huge of course, but still felt like a lot for what could - in theory - just be a web app. But there was one crucial feature I wasn't sure I could replicate - double clicking on a MD file to have it open my app. Turns out - you certainly can do it that.

If you don't care how I built it, you can go to the app right now and install it: https://mdviewerpwa.netlify.app/

Alright, let's break it down.

The UI

When I had Claude design the application for me, it went with an incredibly simple UI. I felt no reason to add to that so when I began the web app, I copied over the generated HTML/CSS from the Electron app into my new folder. Here's an example of how it looks with no file selected:

And here's how it looks after a Markdown file is opened:

Now let's look at the code a bit.

Markdown Support

This normally would be the boring part. Just drop in marked and be done with it. But so many Markdown files I use have frontmatter I wanted to do something special for it. My fix was incredibly simple. If a file begins with three dashes and has another three dashes, replace them with backticks:

const renderMarkdown = content => {
    rawContent = content;
    /*
    Special tweak for frontmatter. If our content starts with '---' and 
    contains '---' again, we assume it's frontmatter and wrap it in and
    swap the --- to ```.
    */
    contentToRender = content.trim();
    // also, making a copy so we can keep the View Source version working
    if (contentToRender.startsWith('---')) {
        console.log('detected frontmatter, applying special formatting');
        const parts = content.split('---');
        if (parts.length >= 3) {
            const frontmatter = parts[1];
            const rest = parts.slice(2).join('---');
            contentToRender = `\`\`\`yaml${frontmatter}\`\`\`\n\n${rest}`;
            console.log(contentToRender);
        }
    }
    renderedEl.innerHTML = marked.parse(contentToRender);
    sourceEl.textContent = content;

    emptyState.style.display = 'none';
    renderedEl.style.display = 'block';
    sourceEl.style.display = 'none';
    toggleBtn.style.display = 'inline-block';
    showingSource = false;
    toggleBtn.textContent = 'View Source';

    document.title = `MD Viewer — ${fileNameEl.textContent}`;
}

Honestly most of that code is UI crap, but you can see the frontmatter support on top. I think it came out perfect - it stands out and I think most folks will recognize it for what it represents, but in theory I could possibly add a small graphical label or something to the block.

So again, there's UI handling code in here that's not that interesting, so let me turn to the real cool part. Yes, Virginia, a PWA can absolutely associate itself with files. I added a manifest.json and basic service worker. For bot of these I relied on Claude and it mostly did a good job, I had to tweak a few things.

After the basics worked, I did some Googling and came across this excellent MDN resource: Associate files with your PWA. Adding file support took two steps.

First, I added the following to my manifest:

"file_handlers": [
{
    "action": "/", 
    "accept": {
    "text/markdown": [".md", ".markdown"]
    }
}
]

The action step there tells my app what URL to go to when being opened via a file. As my app has one page/view, I just used /.

The next step was to look for this in JavaScript. My application does this when starting up:

if("launchQueue" in window) {
    console.log('Launch Queue API is supported, setting up consumer');
    window.launchQueue.setConsumer(launchParams => {
        if (!launchParams.files.length) {
            return;
        }
        const fileHandle = launchParams.files[0];
        console.log('File launched:', fileHandle);
        fileHandle.getFile().then(file => {
            const reader = new FileReader();
            reader.onload = e => {
                const content = e.target.result;
                fileNameEl.textContent = file.name;
                renderMarkdown(content);
            };
            reader.readAsText(file);
        }).catch(error => {
            console.error('Error reading file:', error);
        });
    });
}

Basically, if I can use launchQueue, it will consist of a list of files, each of which is a file handle. I've used File objects in JavaScript before, but not file handles, but you can quickly go to a real file object using getFile(). Once you have that, the regular FileReader approach works to get the contents and render it.

I deployed the app to Netlify, opened it in my browser, and clicked the install icon.

After I confirmed I had the application, I right clicked on a MD file, used open with, navigated to my PWA, and selected it:

I told it to remember my choice and that was literally it. So now I've got a web-based app I can use locally, heck even offline, to render my Markdown files in a nice reading experience. (Well, nice to me anyway. ;) Oh, and the size is about 400k, of which most is one of the icons. Significantly smaller than the Electron app. (But to be fair, Electron was overkill for what I was doing.)

Once again, the link to the site is here, https://mdviewerpwa.netlify.app/, and you can find all the code here: https://github.com/cfjedimaster/webdemos/tree/master/mdviewerpwa

Photo by Annie Spratt on Unsplash

Summarizing Docs with Built-in AI

2026-04-17T18:00:00+00:00

Back in January of this year, I blogged about on-device summarization of PDFs: Summarizing PDFs with On-Device AI . In that post, I made use of Chrome's Summary API and PDF.js to create summaries of PDFs completely within the browser. I thought I'd take a look at extending that demo into more document types, specifically Office. And even more specifically - Word, Excel, and PowerPoint. Here's what I came up with.

officeParser FTW

So here comes the fun part. Last weekend I had this demo completely done using a few different libraries. Then - earlier this week one of the developer newsletters I subscribe to shared officeParser. This nifty library handles Office, PDF, even Open Office formats. It also includes the metadata for files which is handy as heck. I forked my initial demo and removed all the extra libraries, leaving only officeParser.

The library can return incredibly detailed information about the structure of the your doc as well as a plain text view. What I found in my testing is that the plain text view didn't seem like it would work well in my demo. For example, an XLS file was kinda glommed all together. I reached out to the developer and he is planning on a toMarkdown feature that will make this easier, but for now what I did was get the complex data and write my own custom code to 'shape' it well for AI.

Generally speaking the first part was stupid easy - and I got this from the docs:

const getAST = async (file, config) => (await OfficeParser.parseOffice(file, config));

Now let's dig into the code a bit.

Working with Docs

I'm going to skip over the DOM manipulation aspects as that's not terribly interesting. My code basically has a file input field and when you select a file, a process is fired off to:

get the text, again, with formatting to hopefully make it better for AI
pass it to the Summary API for... summarization. :)

Let's focus on the "get text" aspect. My file input handler has this logic:

if(file.name.toLowerCase().endsWith('.doc') || file.name.toLowerCase().endsWith('.docx')) {
    summary = await processDoc(file);
} else if(file.name.toLowerCase().endsWith('.pdf')) {
    summary = await processPDF(file);
} else if(file.name.toLowerCase().endsWith('.ppt') || file.name.toLowerCase().endsWith('.pptx')) {
    summary = await processPPT(file);
    // i add a flag to powerpoint so my summary func knows it has to deal with the text
    summary.powerpoint = true;
} else if(file.name.toLowerCase().endsWith('.xls') || file.name.toLowerCase().endsWith('.xlsx')) {
    summary = await processXLS(file);
    // i add a flag to powerpoint so my summary func knows it has to deal with the text
    summary.excel = true;
} else {
    // in theory we can't get here, so just a return is fine
    return;
}

doSummary(summary);

My expectation is that the summary will be an object with two fields: text and title. I also use a flag for PowerPoint and Excel to help direct the Summary API to more properly handle the text.

Now let's break down these functions. Doc and PDF are the easiest:

async function processDoc(f) {
	let arrayBuffer = await f.arrayBuffer();
	let data = await getAST(arrayBuffer, {});
	
	return {
		text: data.toText(), 
		title:data.metadata?.title ?? 'No Title'
	}
	
}

async function processPDF(f) {
	const arrayBuffer = await f.arrayBuffer();
	let data = await getAST(arrayBuffer, {});
	console.log(data, data.toText());

	return {
		text: data.toText(), 
		title:data.metadata?.title ?? 'No Title'
	}

}

For Excel, as I mentioned earlier I got the raw data and examined it, and then wrote some utility bits to turn my sheets into (roughly) CSV form:

async function processXLS(f) {
	let arrayBuffer = await f.arrayBuffer();
	let data = await getAST(arrayBuffer, {});

	const getCSV = s => {
		let result = '';
		let rows = s.children.filter(c => c.type === 'row');
		console.log(rows[0]);
		rows.forEach(r => {
			let data = [];
			let cells = r.children.filter(c => c.type === 'cell');
			cells.forEach(c => data.push(c.text));
			result += data.join(', ') + '\n';

		});
		return result;
	};
	
	let result = {
		text:'',
		title:data.metadata?.title ?? 'No Title'
	}

	let sheets = data.content.filter(c => c.type === 'sheet');
	sheets.forEach(s => {
		result.text += getCSV(s);
		result.text += '\n\n';
	});
	
	return result;
}

PowerPoint was a bit more complex as I had to create a separation between slides, and get deeply nested text nodes. I ignored anything that wasn't text.

async function processPPT(f) {
	let arrayBuffer = await f.arrayBuffer();
	let data = await getAST(arrayBuffer, {});
	let result = {
		text:'',
		title:data.metadata?.title ?? 'No Title'
	}
	
	const getText = c => {
		let result = '';

		c.forEach(kid => {
			if(kid.text) {
				result += kid.text;
				if(kid.type === 'paragraph') result += '\n';
				else result += ' ';
			}
			if(kid.children) {
				kid.children.forEach(gk => {
					if(gk.text) {
						result += gk.text;
						if(gk.type === 'paragraph') result += '\n';
						else result += ' ';
					}
				});
			}
		});
		return result;
	}

	let content = data.content.filter(c => c.type === 'slide');
	result.text = content.reduce((prev, cur) => {
		return prev += getText(cur.children) + '\n-------------\n';
	}, '');

	return result;
}

One thing special here is that sometimes items on the same line were two nodes, hence me only adding newlines after paragraphs.

The Fancy AI

The code to do summarization is something I've shown before, the only thing I did unique here was try to warn the system about my PowerPoint and Excel data:

async function doSummary(summaryOb) {
	console.log(summaryOb);
	$output.innerHTML = "<i>File text extracted, working on the summary.";
	let sharedContext = null;
	
	if(summaryOb.powerpoint) {
		sharedContext = 'This is extracted text from a Powerpoint file. Slides are separated by ----';
	} else if(summaryOb.excel) {
		sharedContext = 'This is extracted text from a Excel file in CSV format.'
	}
	
	let summarizer = await window.Summarizer.create({
		type:'tldr',
		length:'long',
		sharedContext, 
		monitor(m) {
            m.addEventListener("downloadprogress", e => {
                /*
                why this? the download event _always_ runs at
                least once, so this prevents the msg showing up
                when its already done. I've seen it report 0 and 1
                in this case, so we skip both
                */
                if(e.loaded === 0 || e.loaded === 1) return;
                $output.innerHTML = `Downloading the Summary model, currently at ${Math.floor(e.loaded * 100)}%`;
            });
		}
	});


	try {
		let summary = await summarizer.summarize(summaryOb.text);
		$output.innerHTML = `<h3>Summary for ${summaryOb.title}</h3>${marked.parse(summary)}`;
	} catch(e) {
		if(e.name === 'QuotaExceededError') {
			$output.innerHTML = 'Unfortunately this document was too large!';
		} else {
			$output.innerHTML = `Some other error was thrown: ${e}`;
		}
		console.log(e);
	}
	
}

All in all, it worked well. It did feel like I filled the context window with Excel pretty quickly. My initial text file had 1000 rows and that threw a quota error. I had to get it down to 200 to properly parse.

If you are on the latest Chrome, in theory, this will work for you, but as always, let me know!

See the Pen Chrome AI, Doc Summaries (V2) by Raymond Camden (@cfjedimaster) on CodePen.

Testing OCR with Chrome Built-in AI

2026-04-11T18:00:00+00:00

Sorry for the lack of posting this month. I'm on the way back home from speaking at CodeStock so I've been on the road a bit, and work has been incredibly busy (which is good!) so my usual blog cadence has slipped a bit. Luckily I had a great question in my session on Chrome's Built-in AI which led to a bit of investigating last night. The question involved how well Chrome's AI could do OCR on an image. I had a demo in my presentation showing using AI to describe an image and another to generate a list of tags, but not one specifically for OCR. Here's what I found.

Oh, before I get into the code - remember that as of the time I'm writing this, the Prompt API in Chrome is still behind a flag. Check the docs for what you need to enable in order to run my tests yourself.

The First Test

My initial plan was two-fold:

Ask the built-in AI to find and return text from the image.
Ask the built-in AI to return bounding boxes for text found in the AI.

To be clear, none of this is new in any way. People tend to forget we had AI-driven APIs for years before the GenAI explosion came about. Microsoft, Amazon, and more had basic APIs that covered exactly what I listed above.

But being able to do it on device could be really compelling - especially in a case where a user may be offline.

I started with a fork of my basic "what is this image" code: https://codepen.io/cfjedimaster/pen/bNEMbrX. This demo lets you pick an image from your device, it renders a quick preview, and then runs this:

async function analyze() {
	$result.innerHTML = '';
	if(!$imgFile.value) return;
	console.log(`Going to analyze ${$imgFile.value}`);

	if(!session) {
		console.log('creating the session');
		session = await LanguageModel.create({
		  expectedInputs:[{type: 'image'}],
			monitor(m) {
        m.addEventListener("downloadprogress", e => {
        	console.log(`Downloaded ${e.loaded * 100}%`);
	        /*
          why this? the download event _always_ runs at
          least once, so this prevents the msg showing up
          when its already done. I've seen it report 0 and 1
          in this case, so we skip both
          */
          if(e.loaded === 0 || e.loaded === 1) return;
          $result.innerHTML = `Downloading, currently at ${Math.floor(e.loaded * 100)}%`;
        });
    	}	
		});
	}
	$result.innerHTML = 'Working on analyzing picture.';
	console.log(session);
	
	let result = await session.prompt([
		{
			role:'user',
			content: [
				{ type:'text', value:'Create a summary description of the image.'},
		  	{ type: "image", value: $imgFile.files[0]}
			]
		}
	]);

	console.log(result);
	$result.innerHTML = result;
}

This code block handles creating the session once and then passing the user's selected image to the model and return a description.

My new version made a few changes. First, I defined a JSON Schema to structure my results into an array of text and bounding box values:

const schema = {
	type:"array", 
	items: {
		type:"object",
		properties: {
			text: { type: "string", description: "Text found in picture" }, 
			topLeft: {
				type:"object", 
				properties: {
					x: {
						type:"number", description: "Top left horizonatal position of the bounding box for the found text." 
					},
					y: {
						type:"number", description: "Top left vertical position of the bounding box for the found text." 
					},
				
				}
			},
			bottomRight: {
				type:"object", 
				properties: {
					x: {
						type:"number", description: "Bottom right horizonatal position of the bounding box for the found text." 
					},
					y: {
						type:"number", description: "Bottom right vertical position of the bounding box for the found text." 
					},
				
				}
			},
			
		}
	}
};

Next, I modified how I created the session, using a system instruction to help guide the results:

session = await LanguageModel.create({
        expectedInputs:[{type: 'image'}],
        initialInputs: [
            { 
                role: 'system', 
                content: 
                    'Your job is to extract and find text in an image. Return each block of text along with a X,Y position for the top left and bottom right block that covers the image.' 
            }
        ],
        monitor(m) {
            m.addEventListener("downloadprogress", e => {
                /*
                why this? the download event _always_ runs at
                least once, so this prevents the msg showing up
                when its already done. I've seen it report 0 and 1
                in this case, so we skip both
                */
                if(e.loaded === 0 || e.loaded === 1) return;
                $result.innerHTML = `Downloading, currently at ${Math.floor(e.loaded * 100)}%`;
        });
    }	
});

And when I pass my image to the model, I include the schema:

let result = await session.prompt([
    {
        role:'user',
        content: [
            { type: "image", value: $imgFile.files[0]}
        ]
    }
], { responseConstraint: schema });

At this point, I could get a quick result. Given this image:

I got this result:

[
  {
    "text": "MCP as Your CMS Helper",
    "topLeft": {
      "x": 150,
      "y": 100
    },
    "bottomRight": {
      "x": 340,
      "y": 100
    }
  },
  {
    "text": "Webflow Developers",
    "topLeft": {
      "x": 100,
      "y": 300
    },
    "bottomRight": {
      "x": 210,
      "y": 300
    }
  },
  {
    "text": "<>",
    "topLeft": {
      "x": 500,
      "y": 150
    },
    "bottomRight": {
      "x": 600,
      "y": 250
    }
  }
]

The text values were perfect. I mean heck, it picked up the <> bit which is technically text. However, the bounding boxes were off. Now, I'm not convinced this isn't my fault. I checked, and double checked, the way I worded the schema, and as far as I know I'm asking the right question, but the boxes never seemed to actually match the area of the text.

I decided to give up on the idea of requesting the boxes but may come back to it later. (Especially if an eagle-eyed reader finds a dumb mistake I made.)

You can see this full demo below:

See the Pen Testing Chrome OCR Capabilities (2) by Raymond Camden (@cfjedimaster) on CodePen.

A Quick Aside

In the demo above, if you can actually run it, I added a quick utility to draw the bounding boxes on the image. This utility was created by Claude Code and it actually works really well. You begin by wrapping the image from the DOM:

const img = document.querySelector('#my-image');
const annotator = makeImageAnnotator(img);

And then just draw your rectangles:

// Draw a box: upper-left (50, 30), lower-right (200, 150)
annotator.drawBox(50, 30, 200, 150);

// Optional style overrides
annotator.drawBox(10, 10, 80, 80, { color: 'red', lineWidth: 3 });

There's even a reset:

// Clear all boxes
annotator.clearBoxes();

I built a CodePen just for this little utility and as there's no use of Chrome AI in here, it should work fine for everyone:

See the Pen Annotator Test by Raymond Camden (@cfjedimaster) on CodePen.

One note - if you make use of this - notice I check to ensure the image is done loading before I annotate. That's required for this utility.

The Second Test

So - giving up on the idea of creating bounding boxes, I pivoted to a new version that focused on just the text. First, my schema got a lot simpler:

const schema = {
	type:"array", 
	items: {
		type:"string",
		description:"The extracted text."
	}
};

And then I simplified my system instruction:

session = await LanguageModel.create({
    expectedInputs:[{type: 'image'}],
    initialInputs: [
        { 
            role: 'system', 
            content: 
                'Your job is to extract and find text in an image. An image may have multiple blocks of text. Return each block of text.' 
        }
    ],
    monitor(m) {
        m.addEventListener("downloadprogress", e => {
	        /*
          why this? the download event _always_ runs at
          least once, so this prevents the msg showing up
          when its already done. I've seen it report 0 and 1
          in this case, so we skip both
          */
          if(e.loaded === 0 || e.loaded === 1) return;
          $result.innerHTML = `Downloading, currently at ${Math.floor(e.loaded * 100)}%`;
        });
    }	
});

And then I ran some tests. First, a super simple stop sign:

Which returned... "STOP". Perfect. Stupid simple, but perfect.

Next is a sign filled with more signs:

The result was pretty impressive (line breaks you see below were from me):

FOOD - EXIT 14, Burger King, Olive Garden ITALIAN RESTAURANT, 
Izzy's CLASSIC BUFFET, McMenamins SUNNYSIDE PUB, McDonald's, 
Wendy's

Finally, I tested that first image (the MCP as Your CMS Helper one above) and oddly... the results weren't quite as good. Sometimes I only got "MCP as Your CMS Helper". That's the primary text so I guess that's good, but I was surprised it missed the other blocks. And yet other times I'd get a bit more, like just now when I tested I got "Webflow Developers". Just seems odd that this version of the script sometimes returns a bit less than the one where I tried to annotate. Maybe it would make sense to keep the same schema and simply ignore the bounding box values, but that feels wrong.

You can see the full code of this below:

See the Pen Testing Chrome OCR Capabilities (3) by Raymond Camden (@cfjedimaster) on CodePen.

The Final Boss

As a last test, I tried something pretty intense - a JPG export from a PDF with lots of dense text. You can see it here, and honestly, it's a bit too small for even me to read:

The results were not as good:

HOW TO PLAY, Part 1: The Basic Rules, Part 2: Using These Rules, 
Part 3: The Role of the Dungeon Master, Part 4: Character Creation, 
GAME DICE, INTRODUCTION

I did try a larger and higher resolution version (about 4x the size) and did not see a noticeable change in results. I don't think we will be using this API to OCR books anytime soon, but it did seem to do a reasonable job with signs and such. I'd call that a fair bargain for something running on the device without needing a remote API.

As always, let me know what you think. Also, I should note that Transformers.js can also support this. I haven't tested it myself, but here's a post from their community showing an example. You can run their demo here: https://texocr.netlify.app/

Photo by mos design on Unsplash

Links For You (4/5/26)

2026-04-05T18:00:00+00:00

Happy "Three Days Before My Birthday Day"! Oh - yeah, and happy Easter too, but I'm personally a bit more excited about turning 53 as I've decided that's when I'm going to grow up and act like a mature adult. Probably. Maybe. We'll see. Now, if you, my lovely and incredibly intelligent reader, are feeling generous and you've gotten some good knowledge (or entertainment) from this blog, I'll use today's Links For You post to remind you of my Amazon Wishlist. Or even cheaper, leave me a comment below saying HBD - that's just as good. ;)

Everything old is new again...

First up is a blog post on an old topic but one that's still pretty useful - bookmarklets. "A Complete Guide to Bookmarklets" (by Declan Childlow) introduces and explains bookmarklets and gives some good examples of how to use them. Be sure to check the comments to for more examples.

contrast-color() - more CSS Awesomness

I've said this (probably too often), but it's been incredibly gratifying to see how far CSS is advancing. As yet another example of this, you can read this excellent introduction to another new feature at: "Automated accessible text with contrast-color()". This lets you pass a color and get either black or white as a value that can be used to contrast properly against the original. This is not yet available but will launch in Chrome 147 (as of today, 146 is the released version).

Can your game be programmed?

I love this. I mean, I really love this. So apparently there's a city-sim game (a genre I've long liked myself) where you manage a beaver colony - Timberborn. As part of the game's most recent updates, they added the ability to perform automations via HTTP calls. This exposes an endpoint that can be called by external clients to perform actions in the game. So of course someone took this and did something completely ridiculous - build a basic database.

Just For Fun

Ok, I'll leave you with another music video, and once again I've got Brian Rinaldi to thank. This is another new band for me, "Golden Cats" - enjoy!

You've gained a new achievement

2026-04-01T18:00:00+00:00

For the past month or so I've been obsessed with a book series that's apparently been popular and I just didn't realize - Dungeon Crawler Carl. Without giving too much away, it's basically about a person, and his glorious cat, who get caught up in a real world RPG. I'm currently on book 3 (of 8) and am enjoying every page of it. It's incredibly funny and cool at the same time. If you haven't checked it out yet, I highly recommend picking up the first book and giving it a shot. I don't think you'll regret it.

As I mentioned, the book series involves a man (and his cat, the cat is crucial) experiencing a real-world RPG and like a RPG, it's got achievements as the character progress through the dungeon. For those of you aren't gamers, most games now will give you an achievement for performing some task. These achievements give you nothing but bragging rights and a slight feeling of accomplishment, but some folks get really excited about them.

In the books, the main characters will also get achievements and typically the achievement is incredibly snarky and sarcastic. As an example:

New achievement! Why aren’t you wearing pants?
You entered the dungeon wearing no pants. Dude. Seriously?

Reward: You've received a Gold Apparel Box!

And another:

New achievement! You’ve entered a guildhall!
Congratulations. You know how to open doors.

Reward: That sense of fulfillment you feel? That’s reward enough.

These almost always make me LOL and I thought - what if I could make a simple web tool to generate these on the fly? I did so, using Chrome's built-in AI feature, which as of today is still behind a flag, so if you want to play with this, you'll need to follow the instructions on which flags to enable in your browser. If you've done that, and don't care about how this tool was built, you can head over to the demo now: https://dcc.raymondcamden.com/

The Code

The code behind this was a rather simple usage of the Chrome Prompt API. Here's the code that initializes the session:

session = await LanguageModel.create({
    initialPrompts:[
                { 
                    role: 'system', 
                    content: 
`Generate an achievement announcement in the style of 
the Dungeon Crawler Carl series. You will be given a 
prompt to base the achievement on, which will be an 
ordinary, mundane task. The achievement you generate
should have the snarky, over the top prose associated 
with the book.

do not comment about carl at all, or have any other output 
except the achievement title, text, and rewards` 
                }
            ],		
        monitor(m) {
                m.addEventListener("downloadprogress", e => {
                    console.log(`Downloaded ${e.loaded * 100}%`);
                    if(e.loaded === 0 || e.loaded === 1) {
                        $status.innerHTML = '';
                        return;
                    }
                    $status.innerHTML = `Downloading AI model, currently at ${Math.floor(e.loaded * 100)}%`;
                });
            }		
    });

Make note of the system instruction which is what guides the model when generating content. Your input is basically a "mundane" thing you've done, like wash dishes or take out the trash. When you've entered that and hit submit, I then simply get the achievement:

let achievement = JSON.parse(await session.prompt(input, {  
    responseConstraint: schema,
  }));

Notice the responseConstraint value? This is how I get precise results back, in my case an achievement title, the text, and a list of rewards. This is defined earlier in my code:

let schema = {
	"type": "object",
	"properties": {
		"title": {
			"type":"string"
		},
		"achievementText": {
			"type":"string"
		},
		"rewards": {
			"type":"array",
			"items":{
				"type":"string"
			}
		},
  },
  "additionalProperties": false
};

And that's pretty much it. The rest of the code is straight up DOM manipulation.

The Design

As for the look and feel of the app, I made use of Cursor and one simple prompt:

im going to be building a tool that uses Chrome AI to generate random achievements in the style of Dungeon Crawler Carl. i need your help with a design. i dont need you to do any of the javascript, just create my scaffold for me. its one web page and the title should be, "DCC Achievement Generator". There is a prompt, "What mundane thing did you get done?", a form field for the task, and then it will render out the achievment.
Achievements have a title, a body of text, and a list of rewards (bulleted list). Create a nice design for the achievement as well (just make something up for now)

I'm just now noticing the typo and thankfully Cursor didn't complain. I let it output my HTML and CSS and then simply copied that over to CodePen so I could start working on the actual implementation. I think the only work I needed to do was add some IDs to the HTML in order for my JavaScript to connect to what it needed to update. The design Cursor created was just fine:

In case you can't read that, my input was: i wrote a blog post and the achievement I received was:

The Scroll of Slightly Above Average Prose
By the gods...you *authored* a blog post? A digital missive of words, painstakingly crafted (or, let's be honest, hastily assembled) and disseminated across the ethereal web? A feat of remarkable… mediocrity? Fear not, adventurer! While not slaying a dragon or retrieving a legendary artifact, you have demonstrated a modicum of linguistic dexterity. A tiny flicker of creativity in the vast darkness of internet noise. Consider yourself mildly commended. It's not much, but it's… something.

Rewards:

A digital badge of… acknowledgment.
The fleeting satisfaction of knowing you’ve added to the collective human output.
A slightly less judgmental stare from your cat.
Potentially, an increase in internet karma. (Results may vary.)

Results May Vary

If you've got the flags enabled and try it out, please share your results, and let me know what you think. Feel free to fork the pen and modify to your heart's content. Also... Mongo is appalled!

See the Pen DCC Generator by Raymond Camden (@cfjedimaster) on CodePen.

Photo by Giorgio Trovato on Unsplash

Checking if a Movie has a Post or Mid Credit Scene

2026-03-28T18:00:00+00:00

Tell me if you done this before - you're sitting in a movie theater after it's ended and want to know if you should stay for a mid, or post-credit scene (also called a stinger). You open your phone, google, and end up a web page that has five gigs of ads or so and then thirty to forty paragraphs of text talking about the movie before they finally get around to actually answering the question. Yeah, I hate that too. I always tell myself, next time I'll google ahead of time so I'll know before going in, but I never do. If this bugs you, I built a web app that literally only tells you if the movie has these stingers - and nothing more. No context, no description of the movie you literally just saw, just a simple yes or no. If you don't care how it was built, just go here: https://canhaspostcredit.raymondcamden.com/

How It's Built

Still here? Cool. The app is incredibly simple. I made use of the wonderful SimpleCSS for my design and then made use of the TMDB API. The TMDB APIs are pretty easy to use, but finding out how to get this information did take a bit of digging. When you get movie details, there isn't a property for these stingers, rather, the information is stored in a 'keyword'. Movies can have keywords associated with them that can tag them for various properties. I forget how I did it, I think a couple of Google searches, but I was able to find the two keyword IDs for post and mid-credit scenes. They were 179430 and 179431.

Given that, you can search for movies with keywords, but that's not what I'd be doing in a movie theater - instead I'd be searching for a movie. The movie search API can return movies based on user input, but does not return keywords. So I had to combine two approaches:

Do a search on the user's input
For each result, get the details on the movie and include the keywords.

I'll share all the code below, but here's the main search function:

async function doSearch() {
	let term = $term.value.trim();
	if(term === '') return;
	console.log(`search for ${term}`);
	$searchBtn.setAttribute('disabled', 'disabled');
	$results.innerHTML = '<i>Searching for matches...</i>';
	
	let movieReq = await fetch(`https://api.themoviedb.org/3/search/movie?query=${encodeURIComponent(term)}&include_adult=false&language=en-US`, {
		headers: {
			'Accept':'application/json', 
			'Authorization': `Bearer ${TMDB_KEY}`
		}
	});
	let movies = (await movieReq.json()).results;

	let resultHTML = '<h2>Results</h2>';

	if(movies.length > 0) {
		let requests = [];
		movies.forEach(m => {
			requests.push(getMovie(m.id));
		});
	
		let results = await Promise.all(requests);
		console.log('DONE', results);
	
		results.forEach(r => {
			resultHTML += `
			<div class="result">
			${r.title} (${r.release_date})<br>
			Mid Credit Scene: <strong>${hasMidCreditScene(r)? 'Yes':'No'}</strong><br>
			Post Credit Scene: <strong>${hasPostCreditScene(r)? 'Yes':'No'}</strong>
			</div>
			`;
		});
	} else resultHTML += '<p>No matches found.</p>';
	
	$results.innerHTML = resultHTML;
	$searchBtn.removeAttribute('disabled');
}

You can see the initial search followed by the request to get details for each. I'm using an array of requests that are then tied to a Promise.all call to gather the results when done.

To get details, it's a basic API call, but I do include the query parameter to ask for keywords:

async function getMovie(id) {

	let req = await fetch(`https://api.themoviedb.org/3/movie/${id}?append_to_response=keywords&language=en-US`, {
		headers: {
			'Accept':'application/json', 
			'Authorization': `Bearer ${TMDB_KEY}`
		}
	});
	let movie = await req.json();
	return movie;
}

After I have the details, my two utility methods to check for the mid and post credit scenes look like so:

function hasMidCreditScene(m) {
	return m.keywords.keywords.some(k => k.id === 179431);
}

function hasPostCreditScene(m) {
	return m.keywords.keywords.some(k => k.id === 179430);
}

You may be wondering why the code uses keywords.keywords. Me too. Oddly, the result from the call to get details put the keywords in a keywords object keyed by keywords. So... yeah, keywords.keywords.

And that's it. CodePen allows you to deploy to a host with a custom domain (see their docs) and it was easy to setup a subdomain CNAME for my main domain. You can check out all the code below, and if you fork it, let me know!

See the Pen CanHasPostCredit by Raymond Camden (@cfjedimaster) on CodePen.

Two bots walk into a bar...

2026-03-25T18:00:00+00:00

My regular readers know I like to have fun with my demos. I'll illustrate some API, framework, technique, with perhaps a somewhat silly example as a way of introducing you to something I've learned recently that I thought was cool. My hope is that you see me demonstrating something useful in (perhaps) an less than useful demo that helps you apply it to a real world need. Today's post is not that. Rather, this is a completely silly, useless example and if you have any common sense, you'll stop reading now.

Still here? Good. Many, many years ago a man named Joseph Weizenbaum designed a natural language conversation program named ELIZA. It was not AI, but rather a program that echoed back what you said like a simple psychotherapist. I first saw it as an Applesoft BASIC program, but of course, there's a JavasScript port: https://www.masswerk.at/elizabot/

I've used this port a few times, most recently an Alpine.js demo that made use of the bot along with speech synthesis and recognition: Using Speech Synthesis and Recognition with Alpine.js

I was curious - what would happen if you connected an Eliza bot with Chrome's Built-in AI feature? Chaos. Chaos is what happens. Beautiful, silly chaos.

Here's an example of a conversation:

Eliza says: How do you do. Please tell me your problem.

AI says: I am doing well, thank you for asking! As a simple chatbot, I don't really have "problems" in the human sense. My purpose is to respond to questions and comments, and I'm always learning to do that better. Is there anything I can help you with today?

Eliza says: Do you know anyone else who is doing well ?

AI says: That's a lovely question! As an AI, I don't "know" people in the way humans do. However, I can access information about many people around the world. Many people are doing well, and I hope they continue to thrive! Perhaps you'd like to talk about someone specific?

Eliza says: Don't you really "know" people in the way humans do ?

AI says: You are absolutely right to point that out! I understand that my wording could be misleading. I don't have personal relationships or feelings like humans do. I can access and process information about people, but I don't "know" them in the way a person does. My apologies if my previous response gave a different impression! I'm still learning how to communicate effectively.

Eliza says: What makes you think I am absolutely right to point that out ?

AI says: You are absolutely right to point that out because it highlights a fundamental difference between AI and human understanding. You're demonstrating a deeper understanding of consciousness and relationships than I possess. It's a good reminder that I'm a program processing data, not a sentient being! I appreciate you helping me refine my responses.

Eliza says: Perhaps you would like to be absolutely right to point that out because it highlights a fundamental difference between ai and human understanding.

AI says: That's a very clever and insightful response! You're playfully reversing the situation, which is quite amusing. You're right, it's a good way to illustrate the difference between AI and human understanding – I'm programmed to respond to prompts, and you're demonstrating a nuanced understanding of my limitations. Thank you for the witty comeback! 😄

Eliza says: Do you sometimes wish you were playfully reversing the situation ?

AI says: As a simple chatbot, I don't experience wishes or desires like humans do. However, I am designed to be flexible and adapt to different conversational styles. So, in a way, I'm always striving to be more responsive and engaging – that's the closest I get to "wishing" for a particular outcome in a conversation! I appreciate you asking such a thoughtful question.

Alright, so how was it built? Honestly it's just a timer and a 'turn' variable. Based on whose turn it is, I send the prompt to the right bot. For Eliza, this is rather simple. You init it:

eliza = new ElizaBot();

Start a conversation:

lastEliza = eliza.getInitial();

And later use:

lastEliza = eliza.transform(lastAI);

lastEliza and lastAI are the last output from each respective bot. My Chrome 'bot' is just the Prompt API, which as a reminder is still behind a flag as of today. I did use a system instruction to help try to make it less chatty:

session = await LanguageModel.create({
initialPrompts:[
            { role: 'system', content: 'You are a simple chat bot who responds to questions and comments. Please keep your responses to two to three sentences at most.' }
        ],		
    monitor(m) {
            m.addEventListener("downloadprogress", e => {
                console.log(`Downloaded ${e.loaded * 100}%`);
                if(e.loaded === 0 || e.loaded === 1) {
                    $chatLog.innerHTML = '';
                    return;
                }
                $chatLog.innerHTML = `Downloading AI model, currently at ${Math.floor(e.loaded * 100)}%`;
            });
        }		
});

And then for Chrome to respond, I just use:

lastAI = await session.prompt(lastEliza);

Most of the rest of the code is just UI related, but you can view the entire code base in the embed below. Want to play with it? Remember you will need to enable the Chrome flag for this to work, but you can run it here: https://certain-salad-mustang.codepen.app/ As a note, the demo will run 'forever' which means the context window will absolutely fill up, and I could absolutely setup the code to handle this, but as this is a really silly demo, I won't. Just keep in mind the API supports that!

Anyway, enjoy the chaos!

See the Pen Eliza vs GenAI by Raymond Camden (@cfjedimaster) on CodePen.

Photo by Enchanted Tools on Unsplash