Mozilla rolls out first AI features in Firefox Nightly, and theyre actually useful.
blog.mozilla.org
Tldr: Theyre adding an opt-in alt text generation for blind people and an opt-in ai chat sidebar where you can choose the model used (includes self-hosted ones)
I like how it is opt-in.
Self-hosted and locally run models also goes a long way. 90% of LLMs applications don't require users to surrender their devices, data, privacy and security to big corporations. But that is exactly how the space is being run right now.
And yet, Mozilla went for the 10% that do violate your privacy and gives your data to the biggest corporations: Google, Microsoft, OpenAI.
What happened to the Mozilla Manifesto?
The alternative is only supporting self hosted LLMs, though, right?
Imagine the scenario: you're a visually impaired, non-technical user. You want to use the alt-text generation. You're not going to go and host your own LLM, you're just going to give up and leave it.
In the same way, Firefox supports search engines that sell your data, because a normal, non-technical user just wants to Google stuff, not read a series of blog posts about why they should actually be using something else.
The alt text generation is done locally. That was the big justification Mozilla used when they announced the feature.
I'm talking about the non-local ChatGPT stuff.
Ah, I missed that alt text specifically is local, but the point stands, in that allowing (opt-in) access to a 3rd party service is reasonable, even if that service doesn't have the same privacy standards as Mozilla itself
To pretty much every non-technical user, an AI sidebar that won't work with ChatGPT (Google search's equivalent from my example previously) may as well not be there at all
They don't want to self host an LLM, they want the box where chat gpt goes
But the alt text generation already leverages a self-hosted LLM. So either Mozilla is going to cook in hundreds of extra megabytes of data for their installs, or people with accessibility issues are going to have to download something extra anyway. (IIRC it's the latter).
We could talk all day about things that Mozilla could add out of the box that would make the user experience better. How about an ad blocker? They can be like Opera, Brave, Vivaldi, even the most ambitious Firefox fork LibreWolf.
But for some reason they went with injecting something into Firefox that nobody was asking for, and I don't think it aligns at all with the average Firefox users needs or wants. Normies don't use Firefox. They use a browser that doesn't raise "switch to Chrome or Edge" messages. And if there was some subset of Firefox users who were begging Mozilla for AI, I never saw them. Where were they?
If it was truly opt-in, it could be an extension. They should not be bundling this with the browser, bloating it more in the process.
AI already has ethical issues, and environmental issues, and privacy issues, and centralization issues. You technically can run your own local AI, but they hook up to the big data-hungry ones out of the box.
Look at the Firefox subreddit. One month ago, people were criticizing the thought of adding AI to Firefox. Two months ago, same thing. Look at the Firefox community. See how many times people requested AI.
The extension API doesn't have enough access for this.
While it is opt-in and disabled by default, this is the real problem.
What are they missing? So far, all they've added is a sidebar and a couple extra right-click menu additions. Both of these are available for all extensions.
If that's the case, then it's pretty great that Mozilla is also the exact company in charge of the extension API.
I have only one extension, and I use it longer than I use Firefox. I also trust the developer a lot more than I trust Mozilla.
Mozilla isn't in charge of the extension API, it uses Chromium's WebExtensions API
No. They are basing their implementation on that of Chrome, but nobody is forcing Mozilla to do this ... So yes, Mozilla is responsible for all the APIs they integrate. Of course.
Yeah, just create an entirely new, incompatible extension engine from scratch for this one feature specifically!
This is absolutely not how any of this works.
While Mozilla implements the WebExtensions API based on the W3C standard, they are not bound to a 100% verbatim implementation. Like other browser vendors, Mozilla has the flexibility to extend or modify the API as needed, as long as they maintain compatibility with the core standard. Adding new APIs or features to the extension system does not require creating an entirely incompatible engine. Browser vendors often add non-standard extensions to APIs, which can later be proposed for inclusion in the next version of the standard if they prove useful. So, Mozilla can certainly add new APIs to their extension system without making it incompatible with the existing WebExtensions ecosystem. This is not difficult to understand.
I believe what most people are concerned about, including myself, was the AI features being enabled automatically and then having to disable it like every other application would do to inflate metrics.
Because this is opt in like it says in the blog I am ok with it there and disabled.
Will you need your own account for the proprietary ones? Mozilla paying for these feels like it couldn't be sustainable long term, which is worrying.
The proprietary ones are free
But what does it DO? How is it actually useful? An accessibility PDF reader is nice, but AI can do more than that
This is great, but again, what for?
A lot of people use llms a lot, ao its useful for them, but its also nice for summarizing long articles you dont have the time to read, not as good as reading it, but better than skimming jt
@Blisterexe @Xuderis It's true, as a researcher, these models have helped me a lot to speed up the process of reading and identifying specific information in scientific articles. As long as it is privacy respecting, I see this implementation with good eyes.
It lets you use any model, so while it lets you use chatgpt, it also lets you use a self-hosted model if you edit about:config
But what does using that in my browser get me? If I’m running llama2, I can already copy and paste text into the terminal if I want. Is this just saving me that step?
Yeah
No, that's not useful at all, but Mozilla refused to listen to the blind community.
Can you elaborate? I would love to learn more about the alternative suggestions
Because good alt text needs to be highly context dependant, so you can't automate it. The better alternatives we have right now are crowd-sourced alt text sites, where volunteers may generate descriptions.
There's plenty of situations where even a contextless generated alt-text is a huge improvement on no alt-text at all
You should better read what the blind community thinks about it instead of making blanket assumptions.
I think you have a very optimistic view on how far crowd sourcing this is going to take us.
BTW, you think web developers aren't already using editors that use AI to generate alt text automatically? AI alt text is going to be everywhere regardless.
Also I'm not saying that's a good thing. It's just an inevitable thing.
Then why respond when I was mentioning its usefulness and that the blind community was not heard by the tech bros.
Why isn't it useful?
He's not blind! What's he supposed to do with such a dumb feature?? /s
Because good alt text needs a lot of context, so it must be done by humans for humans at our current state of tech,
The old 'let perfection be the enemy of good' argument..
Surely this is a step in the right direction?
It's not, listen to the blind community instead of making assumptions (I mentioned that in my first comment).
Why not?