Posts tagged tech
Posts tagged tech
1 note &
I tried [Dropbox]. The promise of working cross device file sync sounded really intriguing, but the final straw was [Elements]. Elements is a great app. I purchased, downloaded, and launched it. I signed up for Dropbox right there, then downloaded Dropbox onto both my laptop and my desktop.
Installation was drag-and-drop. From what I’ve heard, the sync works wonderfully. However, when I first launched the app on my laptop, it asked for my administrative user name and associated password. That’s a bit odd for any app, especially one which does not modify the system in any way, so I investigated a little. Dropbox installs three pieces into one’s system when it first runs.
The first one is the app itself, but when launched it marks two files within the app bundle as set-user-id-root (suid). This is extremely strange and extremely dangerous, from both a stability and a security perspective.
The second is a Contextual Menu Item, which is a plugin which adds an item to the right-click menu. However, in addition to the extra feature added to the context menu, this plugin also injects an additional “plugin” when it loads. This is extremely strange and extremely dangerous, from both a stability and a security perspective. (This plugin was for finder.)
Third, it installs a file named “.dropbox” (w/o quotes) in one’s home folder. The placement of this file is strange (the appropriate places would be ~/Library/Application Support/Dropbox and ~/Library/Caches/Drbox), but not at all dangerous.
Some of these mistakes come from Dropbox’s recent addition of Mac support. That is, they just don’t know any better. On windows, significant stability damage through the use of novel and unexpected code injection is the norm. A carefully designed “plugin” would put no more strain on the system than what comes on the system from the original manufacturer. (I’m not saying that windows is more stable with “plugins,” but that nobody would notice the instability introduced by these “plugins.”) On Linux, suid binaries are entirely normal. In fact, significant parts of the user land live entirely in root space, e.g. the X window system itself. There, the security implications are dealt with through over engineering and careful disclosure, while the stability implications are expected to be dealt with by the system administrator (read: end user).
On Mac, however, stability and security are two of the platform’s primary draws. Those two reasons are exactly why I, myself, use a Mac. Furthermore, as a software developer, I can clearly see solutions to the problems that Dropbox appears to have been attempting to solve. For example, the contextual menu could be provided by a “service”, thereby bypassing the need for that plugin. The code-injected Finder “plugin” could be scrapped entirely and replaced with careful use of custom folder icons, supported by the very same folder-change-monitoring system used to watch for actual content changes. Actual content changes could be provided by the system that Apple built into the platform for that very purpose: FSEvents. This is part of the system that Spotlight and Time Machine use.
I uninstalled Dropbox shortly after my investigation. Perhaps I’ll try it again once they’ve had a change to fix up their Mac support.
0 notes &
IMPORTANT: DOING THIS TOO OFTEN WILL SIGNIFICANTLY REDUCE PERFORMANCE. Furthermore, the equivalent happens automatically. THIS IS A TOY ONLY.
Do you like it when Activity Monitor reports large amounts of free memory, regardless of actual performance? Try this: open Terminal and just type “purge” (without quotes).
OMG WTF BBQ
Pretty neat, eh?
The purge(8) utility is a debugging tool which interacts with the Unified Disk Cache. Basically, Mac OS X (and all modern OSes) keeps recently (or frequently) used files in RAM, since it is fairly likely that they’ll be used again very shortly. This does, in fact, happen more often than not.
Most files are tiny and read once and “never” again. (We’re talking about seconds here, so re-opening the same file tomorrow or after lunch is like talking about how your house will look in the year 2256. Its just as good to say “never.”) These files drop out of the disk cache quickly enough, since they’re closed (programatically) as soon as they’re opened. Other files are enormous (many megabytes) and are rarely merely loaded and closed. Instead, these files are frequently read partially, again and again, and often modified in-place. Often, these files aren’t ever read start-to-finish, and certainly aren’t closed once read. Examples include databases, any sort of for-editing media (audio, video, images, animation, &c.), and even not-for-editing consumer-targetted media files (like a 1.4GB movie downloaded from iTunes).
Most applications interact with files in ways that lets Mac OS X know which category they fall into: open and “discard”, or frequent access. Even when they don’t, Mac OS X is fairly good at guessing.
When an application behaves as though it is clearly within the frequent access category (i.e., working with enormous files that simply couldn’t fit into RAM at all), an app may nevertheless explicitly tell the system that it is really in the other category. Creating a new ZIP or TAR archive, or exporting from source material to ProRes, are some examples of this sort of exception.
The problem comes in when an application is working with those aforementioned enormous files, but neglects to tell the system that it really should be in the other category. The purge(8) utility exists so that a developer or system administrator can debug this situation.
The purge(8) utility merely empties the aforementioned disk cache. That’s all.
Just like all caches, the purpose of the cache is to increase performance. And, just like all caches, emptying the cache is usually pointless and generally unnecessary. Just like all caches, it will clean itself over time. Unlike the cache in a web browser, though, “over time” is minutes not days or weeks. However, just like the cache in a web browser, it is extremely unlikely that you will get any performance improvement by manually emptying it.
In rare situations, however, a developer or system administrator may know specifically of a situation where the built-in heuristics will guess wrong, or at least not guess quickly enough.
0 notes &
I haven’t jumped on the HD bandwagon yet. Here’s Why.
One of the reasons that I don’t use Blu-Ray or choose HD when purchasing from iTunes is that I don’t need it. 480p, that is full-resolution, progressive-scan, DVD-quality, is good enough for me. My television is only 33 inches diagonal. Its widescreen and LCD and all that HD goodness. It’s also full 720p, not that almost-720p 1280x800 nonsense. 480p looks damn good on it. 720p looks better. I can see the difference, but only when I’m looking for it. (Music Videos are an easy way for me to compare since I have both 480p and 720p versions of a few. The difference is apparent, but only when looking for the difference.)
At the end of the day, 480p looks good enough.
I use an AppleTV. This is my AppleTV. There are many like it, but this one is mine. I don’t “hack” it, either. It runs stock, Apple-supplied software only. (I have upgraded the hard drive, but that’s not a “hack” in any real sense.)
The AppleTV happily plays 720p25. Movies, generally shot at 24 fps, play just fine in HD on the AppleTV (@ 720p24). PAL-formatted TV Shows (and other PAL-formatted content) also play just fine since PAL runs at 25 fps.
Unfortunately, American TV runs at 30fps. The AppleTV doesn’t play 720p30. For some unfathomable reason, content which is not, never was, nor ever will be produced for television broadcast, will invariably appear at 30 fps if produced in the USA. Thus, this content won’t play in HD. (Actually, it won’t play at all on the AppleTV, but I’m assuming that I’d be able to get it in 480p (or 540p, see below).)
While this sucks, and I would likely purchase a new AppleTV should Apple opt to release (significantly) upgraded hardware, that’s the way it is.
So, some 720p content plays. That’s a recipe for frustration. While movies should encode to 720p24 just fine, some won’t. The result will be that I’ll have to re-encode, wasting much of my time. Or, I’ll have to instruct my encoder to hit a ceiling at 25 fps. That just bugs the frak out of me. I do not want to mess with framerates. That’s a quick way to run quality into the ground. Either way, I’ll end up with some good-looking 480p content, some “great”-looking 720p content, and some not-great-looking 720p-ish content. That would then lead me to become even more desensitised to the quality of the video and might very well leave me indifferent to high-versus-low quality, meaning that I might not notice when something worse than 480p makes it into my library. That, or it’ll drive me nuts (more likely).
The AppleTV will play 540p at 30 fps. I could just encode HD video to 540p, thus solving my problem with 720p. However, now I’m no longer in magical-HD-land, but I’m above SD-lan, so I’m somewhere in in-between land. Depending on how I deal with 720p30 content, I might end up with good 480p, better 720p, and in-between 540p. Again, leading to the mixed-and-matched frustration I just mentioned.
This might be a viable option, but seems like an awful lot of work for a minor increase in resolution with compromise baked-in from the get-go.
The iPhone and iPod touch both play 480p content just fine. While Apple’s official specifications seem to indicate that they are limited to 640x480, this is not true. Anything 480p (widescreen or 4:3, including anamorphic) will play on any iPhone or iPod touch.
The iPod classic will even play 480p content, but will squeeze it to 4:3 somewhat unceremoniously (by basically ignoring the anamorphic settings). I haven’t tested this in a long while, so its possible that the iPod classic isn’t as flexible as I recall. Its definitely close though. (By close, I mean that it will play 480p, but perhaps there are other limitations that make it somewhat incompatible with the 480p on the iPhone.)
I very much like having the ability to put my media anywhere I like. I like to be able to bring my Podcasts and TV Shows, and even a movie once in a while, with me wherever I go. Admittedly, I don’t do this much, but I do do it sometimes and I value the ability to do so. I would, of course, still be able to bring all my 480p media with me, but the new 720p content would fail to sync. Having to check if something is 480p or 720p before deciding if I want to bring it along for the flight will most likely end in frustration for me and my wife, who is more than happy to ignore any technical explanation I offer as to why Dexter won’t sync.
Now that I think about it, my wife does regularly sync our media to her iPhone, so really this isn’t just a vehicle for frustration but is, in fact, a show stopper.
Until recently, it wasn’t entirely clear that reasonably full-featured ripping/encoding solutions were available for Blu-Ray. While I’m as able as the next guy to recompile my linux-kernel with the appropriate drivers and more than capable to step through the Blu-Ray debugger in order to grab decoding keys, it just seems like an awful lot of work for what I’ve already explained isn’t really all that compelling to begin with.
Nowadays, of course, there are much simpler ways of ripping and encoding from Blu-Ray, but they’re still not as streamlined as for DVD. For DVD, most tools can read directly from disc and encode directly to H.264. Tools like HandBrake don’t do this themselves, but are designed so that you can add one plugin to do that actual DeCSS work outside of HandBrake itself. Its about as simple and Mac-like as you can get.
Blu-Ray, to my knowledge, requires a three step process. First, rip. Then, extract. Finally, encode. The rip itself can be done with Mac BluerayRipper Pro. Once that’s done, a somewhat-shady but nevertheless reputable MakeMKV can be used to grab a given “title set” (a single movie or single featurette) into a single file. (I think that MakeMKV can read directly from the disc, if necessary.) MakeMKV, thankfully, doesn’t do any encoding at all. It merely copies the data into a new container package, for easy access. Finally, HandBrake will happily encode H.264 from the MKV container into a nice MP4 wrapper.
For DVDs, I use RipIt to get a DVDMedia bundle, and then HandBrakeCLI to encode. (As mentioned above, HandBrake can read directly from the DVD if VLC is installed.) I’ve written a set of small scripts which make this amazingly painless. I’d have to either expand or replace those scripts for Blu-Ray, again seemingly much work for seemingly small profit.
My TV doesn’t show 1080p. My AppleTV doesn’t play 1080p. My MacBook Air doesn’t show 1080p, and likely couldn’t play it anyway (or at least not very well). My iPhone doesn’t play 1080p. My Blu-Ray — wait, I don’t have a Blu-Ray player.
In short, its just not worth it for me right now.
In the future, however, I’m sure that I’ll jump on the HD bandwagon. Its just a matter of time. 720p and 1080p seem to have gained popularity at the same time and are often confused with each other (since both are referred to as “HD”). I’m thinking that once its clear what format content will be available in (1080p), and that such content is high-quality, I’ll flip the Blu-Ray switch: buy a new TV, buy a new (upgraded) AppleTV, buy a Blu-Ray drive, buy Mac BlurayRipper Pro, rewrite the scripts, and start encoding to 540p — oh, wait.
HD is an expensive project.
Notes &
FileVault was introduced in Mac OS X v10.3, if I recall correctly. Basically, it replaces one’s home folder with a disk image, which is mounted during login and unmounted at log out. The idea is that the disk image is encrypted, and so therefore your home is encrypted. At first, this sounds like a fantastic idea: real data security. However, there are often misunderstandings about what this does and how it works.
A fundamental requirement of any encryption system is that it fails closed. What does “fails closed” mean? When something fails, what happens? If a door lock fails, then its either stuck open or stuck closed. If stuck open, then you can’t lock your door. If it fails closed, then you can’t unlock your door. The goal of all encryption systems is to fail closed. If anything is tampered with, then the contents are inaccessible. Only a working lock (disk image) with a working key (password) can be unlocked. If it ever failed open, then anyone could just come along, smash the lock with a hammer, and walk in.
So, you’ve probably already realised why this is a problem for users of a consumer operating system on a consumer computer. If you drop your laptop, then your hard drive is damaged, then your lock is damaged, then it doesn’t matter if you still have your key. Worse, what if you lose your key? Worse, what if your lock breaks by itself during one of the myriad occasions when a consumer operating system or a consumer computer fails by itself? (I’m not talking hardware failure, but simple application crashes, kernel panics, unexpected sudden power loss, &c.)
The result is that, unless you have some reason to be significantly concerned about data theft, your concern for data loss ought to outweigh all possible motivation for turning FileVault on.
Sadly, we are talking about consumers here. These people don’t backup. Their fear of windows viruses produces an unreasonable desire to turn on anything labelled “security”.
Therefore, FileVault is often a cause of complete, catastrophic data loss.
Well, if you do backups, for example using Time Machine (properly), and you don’t forget your password, and you understand that encryption systems are designed to fail closed, then there aren’t really any down sides, even though FileVault is admittedly often a cause of catastrophic data loss.
I turned on FileVault because I don’t want anyone to get my data if they steal my computer. I don’t want my login details (even though the passwords are encrypted) recovered from my keychain. I don’t want my financial details recovered. I don’t want my personal files recovered (not just pr0n, but also personal e-mails &c.). I recognise that I must now log out in order for my backups to happen. I recognise that I must be more careful about where I save enormous files (such as my iMovie Events folder or disk images of any software I purchase). I recognise that even if I’m extremely careful, my home might be corrupted anyway and I’ll have to restore from backup.
OK. It’s worth it. I don’t want anyone to be able to report what brand of pr0n I like. Not just because of the embarrassment it may cause, but because of the misguided social constructs which govern any sort of public behaviour and which have been enshrined in the ethics requirements of my profession. (I have a whole other essay on why “ethics” requirements are anything but.)
Postscript: Of course, Apple has taken steps to ensure that FileVault is less troublesome. Starting in Mac OS X v10.5, Apple introduced a type of disk image called a “sparse bundle” which breaks the disk image itself into hundreds of tiny files so that the chance of file corruption causing complete data loss is much lower. The sparse bundle format also serves to separate some of the metadata from the file data to improve fault tolerance.
