xaitax/TotalRecall: This tool extracts and displays data from the Recall feature in Windows 11, providing an easy way to access information about your PC's activity snapshots.
github.com
this rootless Python script rips Windows Recall's screenshots and SQLite database of OCRed text and allows you to search them.
Wow, it's pretty wild they didn't even attempt to encrypt or protect this data, even if it is local to your machine. What a treasure trove for malware to sift through.
It IS encrypted. Not well, but it's encrypted.
I thought that it was encrypted if your home directory was encrypted? The impression that I got was that it was just a SQLite database stored in the clear. The user must certainly be able to make queries of that database in order for it to work, so even if it's hosted by a non-user service, malware running locally will still be able to exfiltrate the data.
All true, which is what I meant by "not well" encrypted. It's technically encrypted, but for all practical purposes it might as well not be.
Is it? I skimmed the GitHub source code and couldn't see anything involving encryption, but it's totally possible I missed something. Perhaps just accessing the database from python is enough to decrypt it.
Now ransomware hackers can sell all your shit to someone else if you refuse to pay.
threats to out your porn habits just got more real
Please go through the FAQ section of the git project. It's an eye-opener.
It's worst than that (as bad as this is)...
Today getting some data on a user is bad as smart hackers can put together the context ... However any guessing the hacker has to do may alert the user before the hacked data can successfully be exploited
Now, a hacker would know exactly where each password goes and worse, they'd could learn the entire workflow of internal systems to successfully imitate a trained user...
This means the hacker could use the stolen bank data and legitimately issue credit cards to anyone they want (for example)
It's no longer "we'll expose some data", now it's "we can use this data to infiltrate your systems and wreak havoc in whatever way we want"
I doubt that. Itâs preinstalled and enabled for personal users.
Even if it is enabled by default on pro/enterprise, there will probably be a group policy to disable it.
It feels like this was intended for buisnesses to monitor for phrases on your screen like "coolmath games unblocked free"
or to extract and upload a summary of what happened every second of every day to the server defined in the group policy.
At the same time, I think I would actually use this tool if it were available on a corporate machine.
I already assume my employer is recording everything I do. So, I have zero expectation of privacy.
I regularly have like 30 different tabs and text documents open that meant something to me a couple days ago, but Iâve since lost the context. Being able to ask my computer âwhy was this important?â or âhow did I find this?â would be super useful.
I doubt it. There are plenty of tools that already do this if that was what they wanted, theyâd just model it after those. Storing it locally isnât how such tools usually work, they get shipped off to a remote server for ingestion.
For the kids
Sony BMG copy protection rootkit scandal
Morons:
Thatâs wild. Iâm surprised I never heard of this. Straight up malware.
In a just society the Sony execs would have been jailed for CFAA violations.
Very specific, but makes sense
OBJ to you too, friend đââïž
Hilarious to me that it OCRs the text. The text is generated by the computer. It's almost like when Lt. Cmdr. Data wants to get information from the computer database, so he tells the computer to display it and just keeps increasing the speed --- there are way more efficient means of getting information from A to B than displaying it, imaging it, and running it though image processing!
I totally get that this is what makes sense, and it's independent of the method/library used for generating text, but still...the computer "knows" what it's displaying (except for images of text), and yet it has to screenshot and read it back.
It happens the same on android for some reason
Like 5-8 years ago the google assistant app was able to select and copy text from any app when invoked, I think it was called ânow on tapâ. Then because theyâre google and theyâre contractually obligated to remove features after some time, they removed this from the google app and integrated it in the pixel app switcher (and who cares if 99% of android users arenât using a pixel, they say). The new implementation sucks, as it does ocr instead of just accessing the raw textâŠ
It only works fine with us English and not with other languages. But maybe itâs ok as it seems that googleâs development style is us-centric
Now on Tap also used OCR. Both Google Lens and Now on Tap get the same bullshit results on any languages that are not Latin. Literally, Đ gets read as >|< by both exactly the same.
They changed it, in the beginning it was using the text and not ocr
For example this app could be set as assistant and get the raw text https://play.google.com/store/apps/details?id=com.weberdo.apps.copy
But only the app set on system as assistant can do it
I was very disappointed when they changed it around 2018 as it produced garbage in my language when it was working so goodâŠ
Hey, yeah⊠why arenât they just tapping the font rendering DLL?
âŠare they tapping the front rendering dll??
My guess is that they looked at their screen reader API, saw that it wasnt 100% of the text on screen and said fuck it! Were using OCR!
Having worked on a product that actually did this, it's not as easy as it seems. There are many ways of drawing text on the screen.
GDI is the most common, which is part of the windows API. But some applications do their own rendering (including browsers).
Another difficulty, even if you could tap into every draw call, you would also need a way to determine what is visible on the screen and what is covered by something else.
That's the thing, it doesn't really know what it's displaying. I can send a bunch of textboxes, but if they're hidden, or drawn off-screen, or underneath another element, then they're not actually displayed.
Text from OCR is one kind of match. Recall also runs visual comparisons with the image tokens stored.
To be fair, Data was designed to be like a human, and was made in the image of his creator. He has a number of design decisions that are essentially down to his creator wanting to create something like a human. Including that which you describe.
Data was never intended to work like a PC, it's very normal that he can't just wirelessly interface with stuff.
Wait so the malware won't even need a rootkit first? Damn this is worse than I thought...
the screenshots and text are just sitting in the appdata folder, which requires no special permission to access
Nice đ having extra pw manager n stuff in secret encrypted file only temporary handle decrypted PWs in RAM etc. But then, if you accidentally click on the eye, boom screenShot PW saved as pic of clear Text, nice. Also all personal eBanking stuff etc. And of Course, if you stream Netflix, tons of copyright protected material, lol.
Nope, DRM protected content like Netflix is one of the few things it doesn't capture, it's even mentioned in Recall's privacy section. I'll admit that that's likely due to technical reasons with how the video stream is decrypted and decoded on the GPU and is never actually accessible to the user, not necessarily because they wouldn't want to save that as well.
Malware won't even need to wait for the user to access something sensitive, they can just go back through the user's Recall history and get the data for immediate exfiltration. No chance for anti-malware software to update and catch it before it does anything truly bad, it will just always be too late if given even a minute.
Imagine how easy is the life of law enforcement now.
Before if they seized a laptop encrypted with bit locker they could not do anything.
Now they just need to ask Microsoft the encryption password, which is automatically and silently saved in the Microsoft account (now mandatory) and they can have all the history of what the subject of the investigation did in the past years
What? Bitlocker key tied to MS account and mandatory? What's the point of encryption if the key isn't secret any more?
To protect against casual theft of a device causing the data to be in the thief's hands in addition to the actual device.
The average person unfortunately is not likely to properly backup their encryption keys so if they forget their password (or don't use one and rely on the default of just TPM), they'll complain about losing their data. Having the key backed up gives them a way to get their data back in non-theft situations.
I like how people on lemmy seem to only think of the high stakes state sponsored theft. And not the theft thatâs thousands of times more common.
Yeah. Most theft targets the hardware, not the data within.
Ok, I can saw value in that but why mandatory? While most doesn't backup their keys, I do and I don't need MS help.
On top of the reason the top level comment gave (easy for law enforcement) it also allows for better data collection (linking your activity to your account, no matter where, how or when it is recorded)
Itâs secret to most, not all.
In a hilarious and infuriating side note, MS is obviously doing their absolute best to blame-shift here.
Itâs code. Itâs a project someone made to graphically illustrate and demonstrate, in the wild, why the entire concept of MS Recall is an absolutely awful, foundationally-flawed idea. It is not a âhacker toolâ. The MS c-suite and board members are just pissed that stock go down as a result of their stupidity, and theyâre looking for people to blame who arenât themselves.
There is not a single word in that article that says anything about blame shifting. That title was written by wired.com
Where is the blame shifting? The article says they made no comment and the only MS quotes are just random pr feature blurbs
Dude the headline:
Itâs absolutely not a âhacker toolâ. Itâs a proof of concept. Itâs just code. The author and/or editor is leaning on ingrained negative kneejerk reactions from less knowledgeable members of the general public towards the term âhackerâ.
So that's not Microsoft, that's Wired doing that. Also it IS a hacker tool. It's a tool to automate the scraping of data and sending it somewhere.
He's a white hat hacker, releasing the tool to raise awareness. If he was a black hat hacker he'd be holding onto it and praying Microsoft goes through with release so he could use it to compromise systems.
I don't see any blame shifting at all
How could the db be all plaintext unencrypted?!? I mean this is amateur hour at display here
How are they supposed to feed it into their LLMs later if it's encrypted??
Decrypt it server side like all other encrypted data
If we believe it doesnât leave the machine then the ai can have a decryption layer
That takes up precious cpu cycles
So does the rest of it
If only Microsoft required a second prossesor like some sort of module just for encrypting and decrypting things without using additional CPU cycles... What if we also stored the encryption keys on that module so we could trust that platform...
Honestly I'm pissed that even if I switch OS I'm probably going to be paying more for CPUs from now on to account for microsofts blatant abuse of a monopoly.
How old of a system are you running because TPM have been included on CPUs since at least 2009. Microsoft requiring something already built into modern CPU isn't the reason why CPUs cost more now.
First off. Windows 11 requires TPM2.0 introduced in 2014. Second, the first consumer cpu to include ftpm wasn't until 2015.
The version that windows requires does not matter as I was making the point that we've been dedicating silicon for TPM for a pretty long time now and that there's no corelation between Microsoft's requirements and the recent CPU cost increase.
-quite literally the book on Trusted Platform Module.
Doesn't change that cpus haven't come with ftpm packaged until tpm 2.0 which was barely a decade ago. If you wanted TPM before, you had to have a motherboard with a tpm header and purchase a proprietary tpm module, even then most were only for 1.2, so even then you cant make an older cpu compatability with win 11.
So . . . MS wants to force Recall on us.. Assures us that it's "secure." And it can't be bothered to even lightly encrypt the data? This is just plain incompetent.
Also, MS want to bundle CoPilot with Office 365, a subscription service. You will be paying for the privilege of spying on yourself.
What an unexpected turn of events.
Imagine if they zero day this.
Lol "if". This thing is going to be a massive target.
Someone has already demonstrated using an off-the-shelf infostealer to steal the Recall database from a test computer. It won't take any special skills or technology for this to be a problem.
I was gonna make a joke on how there's no root on windows, but then I remembered sudo for windows is now a thing so.......
sudon't please --prettyWindows be like
good luck to people typing their passwords in visible mode
Windows, pretending it can't read what you're typing in because you didn't click "show password":
That was quick.
Is this thing even a thing? I see lots of discussion about it, but not real proof it was enabled across the board on all win 11 computers.
Yes. It's very very real and is just a typical Microsoft bullshit move.
https://x.com/GossiTheDog/status/1798812975735996772
it's on "Copilot+" PCs (i.e. ARM-based with an NPU)
Iirc chrome stores your local cookies/session in a place malware could also attack. Probably the same idea for other browsers.
I'm not sure I fully understand the issue here. If we're ok with that info being trivially retrievable by a bad actor, why isn't this ok?
Like I get you may not like it, and it's a target, but there are already lots of targets that have gotten a pass based on user permissions. Is it just the breadth of potential info? With the cookies you could potentially log into someone's bank account.
browser data is a potential liability, sure, but you have tools to manage it. you can delete pages or entire websites, you can use private windows, you can purge history older than 6 months or something like that, and at least a few browsers have a "forget" button that wipes out the last two hours of history. similar deals with cookies and other data, and we've collectively decided the benefit of having browser data is worth the risk.
not so here. Recall is a record of everything you've ever done on your PC. you can't selectively delete things like you can with browser history, the app and website exclusion is only as good as whatever Recall is using to detect apps and websites, and you can't redact sensitive info after the fact. people are generally okay with browser history and data because they know they have fine-grained controls to manage it, controls Recall doesn't have
So if they had a ui with buttons to 'pause for X length (could be forever)', buttons to 'forget last X length (once again could be forever), but everything else stayed the same, would it be acceptable?
Like I'm genuinely curious here.
if i were designing a recall program, here's how i would do it: it would take a screenshot every five seconds, OCR it, then run it through local quantized image recognition and word association neural networks, and then toss everything into a CryFS vault. when launching the recall program, you have to provide the password to unlock the vault so it can read and write to it. it can only run in the foreground (so you have to keep the window open for it to run, no closing it and forgetting about it) and it will display a status indicator in your system tray that provides a menu to pause or stop recording. afterwards, you can mark any text or region of the screen for redaction, and it'll redact it across all screenshots and delete it from the database; you can delete individual screenshots or entire periods of time; and there will be an easily accessible self-destruct option that shreds the database (i.e. overwriting it with random garbage 21 times before deleting it off the disk). this is all offline and the application will not request network access
i'm just making this up on the fly, so there are absolutely security and privacy considerations I absolutely forgot about, but this is the bare minimum i would like to see
IIUC it wouldn't be able to be automatically started then, right? I mean I guess you could drag it to startup but it would need the password to start. From a security minded perspective that's good, but from a user perspective kind of sucks. I already unlocked the computer: as a user id just want it to 'work'.
There is always a tug of war between best level of security and user experience. I guess the best security is to get rid of the human element though.. so eh.
Always forced to foreground makes it even less convenient and kind of odd. I dig the status tray control though. I don't see this functionality as being useful if you have to remember to turn it on. If I remember what I was doing enough to turn it on, I'd write down what I'd forget. To me it's about allowing the user to pick their comfort level.
I figure the cryptfs could be a bitlocker volume with a different key than the base C drives key to get similar protection. In theory it could also be based on the C drives bitlocker for a less secure, but still hardware level secured middle ground. Id have to think about it more.
The other stuff mentioned is basically what it does locally in terms of OCR and recognition.. just with proprietary local recipes.
Thanks for your thoughts.
that's true, but since this is a record of everything you've ever done, i feel this is the irreducible minimum for security. a separate password prompt would signal to the less technically-minded users that this is Serious
this is a design pattern i borrowed from Linux (my OS of choice). modern Linux apps require your explicit permission to run in the background, so most of them don't even bother with running in the background at all. that said, i suppose it can run in the background, as long as the status indicator is sufficiently noticeable, but you'd have to go into the settings and flip that switch yourself
i imagine that it would become a habit, or you'd set it to run on startup. my use case would be turning it on for specific tasks like research or shopping, where you might only later remember that that one thing you saw was actually really valuable
can a user-installed app do that?
When you go on the internet you are accessing content on other people's computers. You are saying, "I want such and such document". There's an inherent lack of privacy in browsing the internet. You can try to be private about it, but ultimately you're not changing that you're requesting data from other people's computers and sending them data.
When you are doing something else on your PC besides browsing the web, Recall is still taking screenshots and tracking you. What apps you use, pictures you view, and many other things that might be completely offline and you don't necessarily want a history of stored on your PC, with screenshots and searchable summaries. Do you want each and every one of your fap sessions recorded? Why would you want any of your offline activity recorded?
What if you forget to pause this feature and someone finds these screenshots? Who cares, right? What if your a closeted gay teen living in a conservative country and your family finds the history?
Then there are people who don't understand computers using offline business software for accounting, or whatever, and even if they store their data files on an encrypted drive or something, Recall is taking screenshots of everything they do. If they don't even know its happening, their PC could have years of data that could be stollen from them at any point in the future. Even if they never open those encrypted files again. Obviously, if their computer is pwned, then the hackers could just take the enencrypted files when they're next accessed, but Recall snapshots everything all the time, even if you delete it.
Edit a self nude photo on your PC and forget to turn off Recall, and then layer decide to delete the photo... Too bad, Recall still has it.
It's a feature that's... ok if you want it, but it should not be part of the operating system, and it definitely shouldn't be opt-out. It should be an app that you install with deliberate purpose if and only if you want itand understand the security and privacy risks.
Microsoft instead wants to install it by default and probably turn it on by default. Even if it ends up being opt-in, MS has a long history of asking people to enable features in misleading ways. And the vast majority of Windows users don't understand computers!
I tend to agree with a lot of what is said here. Though it is (assuming they're honest) local only to be clear.
If it was an opt in feature with robust configurations including encrypting the db based off your login session and was auto locked up on log off/reboot (until login again): is that good enough, or would folks then say we should assume the account is also compromised?
I'm trying to disambiguate between generalize ai dislike, Microsoft dislike, windows dislike, distrusts, etc. to consider a world where this exists in Windows and people who would use the feature would feel comfortable
In other words, consider an app that did the same thing. What security constraints would be expected?
First, false equivalency.
Second, weâre not okay with cookies and session being in a place that could leak â itâs why weâre doing everything possible to stop that from happening (I mean GDPR alone is one effect of this).
Third, the fact that you canât see a difference between cookies, which actually can be secured via proper encryption and signing, and a literally unencrypted database driven by OCRed screenshots (taken every couple of minutes) that requires an opt-out and is a very small slippery slope to that data making its way back to Microsoftâs own servers for their own greedy pursuitsâŠ.then Iâm not sure what to tell you.
Recall is wrong. And itâs indefensible. Period.
If you think itâs okay, then feel free to open everything up to Microsoft of who you are and what you do on your Copilot+ PC. I, for one, among many, will choose to secure my information as best as possible, including never using another Microsoft product again, if at all possible. And Iâve already done so for myself.
GDPR has little to do with this. People use site cookies to remember sessions and not have to login again, etc. I'd guess most browser users use and want to use this functionality. If you're fully opting out to not even have persistent sessions, I'm guessing you're in the far minority of users here.
I'm not aware of any non-trivial readily available built-in encryption for cookies. There are easy to find libraries that exist to just pull out cookies (stored locally including session tokens).
To clear up a bit more misinformation from your response: this is an offline feature. The data doesn't go back to Microsoft. It works even if your computer is disconnected from the internet. If you consider their word to be a lie on this part, that's you're right to believe, but until proven, isn't a fact.
Not at all true, GDPR is the exact reason why you see all of the sites these days letting users know that their site stores cookies and requesting acceptance of it. Hence why I said we, as a global society, are trying to do something about this, even if it's something as simple as cookie use disclosure on sites -- it's a start.
Never once said I did.
You're correct, data-at-rest encryption doesn't exist for cookies, but data-in-flight does with SSL. Also, signing cookies and samesite origin is a thing being done these days, which makes them quite improbable, if implemented properly, to be hacked for any actual use in terms of leaking logins to said sites.
For the moment, that's what they say, yes. And that's the problem, especially since it's turned on, by default. This -- is not -- something -- Microsoft has earned trust for.
But you are free to believe them all you want -- the rest of us who have seen what Microsoft has done these past 40 years use that as a guide to judge -- and history is usually a very good judge.