Difference between revisions of "User:Mjb/Workspace"

From Offset
Jump to navigationJump to search
(Misc: -> Drafts, updated)
(moved more content to separate pages; removing Tahmoa / Microsoft Sans Serif / Windows File Protection (WFP) content)
Line 1: Line 1:
== Drafts ==
+
== Topics ==
 +
=== Music & audio ===
 +
* In 2008–2010, I tried to keep track of DJ [[User:Mjb/Beppe Loda mixes and links|Beppe Loda mixes and links]].
 +
* In 2013, I made some notes regarding [[User:Mjb/RTLSDR|RTLSDR]], which is software-defined radio for a receiver based on the RTL chip.
 +
=== XSLT ===
 
* In 2005, I started writing a report on [[User:Mjb/Oracle XSLT processor bugs|Oracle XSLT processor bugs]].
 
* In 2005, I started writing a report on [[User:Mjb/Oracle XSLT processor bugs|Oracle XSLT processor bugs]].
 
* In 2007, I was writing [[User:Mjb/OASIS XSLT Developer Questionnaire answers for 4Suite|OASIS XSLT Developer Questionnaire answers for 4Suite]].
 
* In 2007, I was writing [[User:Mjb/OASIS XSLT Developer Questionnaire answers for 4Suite|OASIS XSLT Developer Questionnaire answers for 4Suite]].
* In 2008–2010, I tried to keep track of DJ [[User:Mjb/Beppe Loda mixes and links|Beppe Loda mixes and links]].
+
=== Windows 7 ===
 
+
* In 2012, I summarized what I learned about [[User:Mjb/Windows 7 folder sharing|Windows 7 folder sharing]].
= RTLSDR setup =
+
* In 2012, I summarized what I learned about [[User:Mjb/Windows folders that can't be renamed or deleted|Windows folders that can't be renamed or deleted]].
I have a USB dongle with a terrestrial digital TV (DVB-T) receiver in it. It is based on the Realtek RTL2832U chip. Tinkerers have figured out that these devices make for nice, cheap radio receivers that can be used with software defined radio (SDR) apps. Thus, this type of dongle, or the use of this type of dongle in this way, is called "RTLSDR".
+
* In 2012, I summarized what I learned about [[User:Mjb/Setting up Remote Desktop|setting up Remote Desktop]].
 
+
* In 2012, I summarized what I learned about [[User:Mjb/Setting application compatibility mode|setting the application compatibility mode]] for multiple programs at once.
My particular dongle uses the best tuner chip, the Elonics E4000. This chip has a defined range of
+
* In 2013, I summarized what I learned about [[User:Mjb/Java updates|Java updates]] and affected apps like Serviio.
64–1100 MHz and 1250–1700 MHz, and is said to be capable of 50–2200 MHz, which looks about right; I can get it down to 51 MHz in SDR#.
+
=== Fonts ===
 
+
* In 2007, for Discogs, I researched how the [[User:Mjb/Phonorecord copyright symbol|phonorecord copyright symbol]] renders in various circumstances.
(Compare to my new portable radio receiver, the Grundig S350DL, which covers 3–28 MHz, the FM radio band of 87–108 MHz, plus the MW (AM radio) band of 0.525–1.71 MHz.)
 
 
 
As per info and instructions at [http://rtlsdr.org/softwarewindows rtlsdr.org], plugging in the dongle will result in Windows installing a device driver for it. This driver is useless; you have to replace it by downloading the proper driver and then running [https://github.com/pbatard/libwdi/wiki/Zadig Zadig], which is the official GUI front-end for libdwi, a generic installer for USB device drivers.
 
 
 
I didn't notice until too late, but there's an automated installer script at rtlsdr.org which will do most of the initial setup for you:
 
* Download a replacement RTLSDR driver from [http://sdr.osmocom.org/ Osmocom]. You need the 32-bit driver, even on 64-bit Windows (probably because SDR# is a 32-bit app).
 
* Download Zadig.
 
* Download SDR# (the nice GUI radio app) ''and'' its RTLSDR plugin from [http://sdrsharp.com/ sdrsharp.com].
 
* Unzip SDR# into a new <code>sdrsharp</code> subfolder, relative to wherever the script is run from.
 
* Copy the SDR# RTLSDR plugin DLLs and modified config file to the <code>sdrsharp</code> folder.
 
* Unzip the 32-bit RTLSDR driver DLL (even on 64-bit Windows, this is what you need, I assume because SDR# is 32-bit) into the <code>sdrsharp</code> folder.
 
* Unzip Zadig into the <code>sdrsharp</code> folder.
 
* Dispose of temporary files & folders.
 
 
 
However, even if you use the automated installer script, you still have to run Zadig yourself in order to install the RTLSDR driver! Just plug in the dongle and follow the instructions at rtlsdr.org to choose all the right options, then click Replace Driver. (I am guessing Zadig notices the driver DLL sitting in the same directory as zadig.exe).(?)
 
 
 
Now you should be able to run <code>SDRSharp.exe</code>.
 
 
 
Near the top left corner of the window, you choose the input source. It is set to "Other (sound card)" by default, but if all is working, you should be able to set it to "RTL-SDR / USB". This option won't appear in the list if the plugin didn't get installed, and choosing it will produce an error if the driver didn't install or if the dongle isn't plugged in.
 
 
 
Then press Play... and have fun.
 
 
 
= Windows 7 related =
 
== Folder sharing ==
 
This is confusing as all hell.
 
 
 
Ordinary '''file and printer sharing''' in a Windows workgroup or domain is really just '''randomly shared folders''' (and maybe also printers), and it is available in all versions of Windows. In this system, any folder can be designated as "shared" and will thus be potentially visible to ''everyone'' on the LAN. However, each person's actual ability to see, list the contents, and read and write files in a shared folder on an NTFS-formatted drive depends on that folder's explicit or inherited permissions, as seen in the Security tab of its Properties window.
 
 
 
The designation of folders as shared is something that happens on the drive, in the folders themselves. So if you attach the drive to a different computer, the folders will be shared on that computer, too!
 
 
 
The somewhat-separate concept of a '''Homegroup''' (or HomeGroup; they aren't consistent with capitalization) is like a password-protected workgroup in which other members of the group can access each some or all of each other's '''Library folders'''. This feature was introduced in Windows 7 and is not supported in older versions of Windows.
 
 
 
Basically, the Library folders, or some designated subset thereof, are visible in the Homegroup section of Explorer on all the LAN computers configured with the same Homegroup password. This also results in a bunch of clutter in the ordinary file and printer sharing space: the folders that are in the Library folders are visible to Homegroup users (HomeUsers group) as ordinary shared folders at the root level, and the Library folders themselves are visible at Users\Whoever\AppData\Roaming\Microsoft\Windows\Libraries.
 
 
 
The Homegroup password can be viewed and set in the Homegroup control panel (just type Homegroup from the Start menu). If you join a new LAN and want to be part of the Homegroup, you must change the password in your Homegroup settings so it matches the one used by the group you want to join.
 
 
 
Some experimentation reveals that symlinks (introduced in Windows Vista) are honored by the Libraries, but only for local browsing in Explorer. Other Homegroup users can see the links but can't follow them. For example, I have a music folder in which some folders are real and some are symlinks (made at the command line with <code>mklink /D</code>), so I can browse the folder kind of like a library. When the folder is added to the library, it works fine; I can follow the links as I browse the library. But if I am connected from elsewhere on the Homegroup and am trying to browse that library, I can't follow the links.
 
 
 
=== Master control ===
 
 
 
There's an on/off switch for all folder sharing here:
 
 
 
<tt>Control Panel\Network and Internet\Network and Sharing Center\Advanced sharing settings</tt>
 
 
 
It's the File and Printer Sharing control, but I believe it also affects Homegroup sharing as well.
 
 
 
That control panel has separate sections for each type of network (home, work, public, whatever), so make sure it's set the way you want it in each place.
 
 
 
=== The lock icon ===
 
 
 
When you create a folder, normally it won't have any explicit permissions on it; it will just inherit permissions from its parent. Thus, if you move the folder elsewhere, it will inherit new permissions depending on where you put it.
 
 
 
If you haven't set any custom permissions on any ancestor folders, then the default permissions that will be inherited from the root, as seen in the folder properties Security tab, will be as follows:
 
* SYSTEM - full control
 
* Administrators - full control
 
* the owner - full control
 
* Users - list contents, read & execute, read
 
* Authenticated Users - same as Users, plus modify and write
 
 
 
If the folder is explicitly shared with the Homegroup, it will have HomeUsers permissions as well:
 
* HomeUsers - for Homegroup (Read) - list contents, read & execute, read.
 
* HomeUsers - for Homegroup (Read/Write) - full control.
 
 
 
These Homegroup users won't see the folder in the Homegroup section of Explorer on their machines, though; it will only be in the Network section, with the rest of the normally shared (non-Library) folders.
 
 
 
'''The lock icon simply means the folder does not have read permissions for Users, Authenticated Users, or HomeUsers.''' The icon is telling you that the folder is only be accessible to certain designated people (like the owner) and Administrators, regardless of its shared or unshared state.
 
 
 
Just because a folder has the lock icon doesn't mean it isn't shared—it could still be included in a Library which is shared with the Homegroup! IIRC, what happens is they can see the folder in their library, and they can even see its contents, but they can't access the content. (I need to verify whether this is true, though).
 
 
 
See below for how to remove the lock icon. As I said, it's a permissions issue, nothing to do with sharing.
 
 
 
=== What folders are shared? ===
 
 
 
====Homegroup====
 
To see what Library folders are shared with your Homegroup, you can look at your own Homegroup in Explorer, or (better) type Homegroup in the Start Menu to go to the HomeGroup control panel.
 
 
 
====Regular folder sharing====
 
 
 
=====Explicitly shared folders=====
 
Go to Computer > Manage, and under the System Tools, look at Shared Folders. These are things that are shared and probably are visible to everyone on the local LAN (e.g., the wired or WiFi network in your home, office, or public place).
 
 
 
Ideally, in the management console under Shared Folders, you should only see the default shares: one for each drive's root folder, plus the Remote Admin and Remote IPC shares. Don't worry; although these can't be unshared, they don't show up when other people are looking at your shares, and even if someone attempts to access these folders by name, they won't get in unless you've set special permissions or given them your Windows username & password.
 
 
 
If you see other folders shared here, then it means that they have been explicitly shared, regardless of what permissions are set on the folders themselves.
 
 
 
For example, if you see a share named "foo", and your computer's name is "ROCKSTAR", then it means the UNC path <tt>\\ROCKSTAR\foo</tt> is accessible to everyone on your LAN, unless it was further configured with special permissions to lock people out. You can test it by connecting to browsing the Network; it's linked from the bottom left corner of the folder pane in Explorer on Win7. Try it from different computers.
 
 
 
=====Public folders=====
 
Unless you have turned it off, '''Public folders are also shared'''.
 
 
 
These folders are located in <tt>%PUBLIC%</tt>, which is usually <tt>C:\Users\Public</tt>. Sharing of this folders can be controlled under <tt>Control Panel\Network and Internet\Network and Sharing Center\Advanced sharing settings</tt>, which you can also reach via the "Share with" context menu item. That control panel has separate sections for each type of network (home, work, public, whatever), so make sure it's set the way you want it in each place.
 
 
 
Sharing public folders will actually also result in C:\Users being shared, too. People will therefore be able to see the user folders of whoever has shared folders or libraries. Only whatever's actually shared within them will be visible.
 
 
 
If you share the Public folder via the control panel, then unshare it by some other means, the control panel will still say that the folder is shared. If you do really want to share it again, just set the control panel to not share, save changes, set it back to share, and save changes again.
 
 
 
=== Sharing and unsharing a folder ===
 
In Explorer, if you highlight a single folder, you have the option of choosing "Share with" from a drop-down or context menu. This will change sharing ''and'' permissions:
 
 
 
* Nobody - unshares the folder, and deletes Users, Authenticated Users, and HomeUsers permissions. Does not restore inherited permissions.
 
* Homegroup (Read) - shares the folder, converts inherited permissions, and adds HomeUsers permission (read only).
 
* Homegroup (Read/Write) - shares the folder, converts inherited permissions, and adds HomeUsers permission (full control).
 
* Specific People - shares the folder, converts inherited permissions, and adds appropriate permissions for whoever you choose. If you choose Everyone, it adds permissions for Users and Authenticated Users.
 
 
 
You can also get to the 'Specific People' dialog by pressing the Share button in the Sharing tab of the folder Properties.
 
 
 
Alternatively, from the right-click menu in the Shared Folders management console, there is a Create A Shared Folder Wizard. This prompts you for a folder to share (or you can make a new one), a share name & description, offline settings, and gives you several more options than usual for Share Permissions. (I don't understand actually understand the difference between Share Permissions and other permissions...)
 
 
 
=== Permissions ===
 
 
 
When you share a folder, its inherited permissions are converted to explicit ones. When you unshare the folder, do you want these explicit permissions to remain, or do you want to get the folder back to its pristine state with only inherited permissions? It depends on the situation; I can't tell you one is better than the other.
 
 
 
* Since I usually want inherited permissions, my preferred way to unshare a folder is to create a new folder as a sibling of the shared folder, move the shared folder's contents over to it, delete the shared folder, and rename the new folder. The new folder will not only be unshared, it will also have the default, inherited permissions like you probably want. '''This is also a good way to get rid of the lock icon.'''
 
* If you want to temporarily unshare a folder, you can just right-click on it in the management console under Shared Folders, and choose "Stop sharing". This will just stop the sharing; the permissions won't change. So if you want to go back to inherited permissions, you have to go in and delete the explicit permissions in the Security tab. It's not that hard, but not obvious or something you want to mess up.
 
* Another way to unshare a folder is to choose to share it with Nobody, or with just the owner. But again, this leaves explicit permissions on the folder, and furthermore will result in a lock icon. To get rid of the lock icon, you could share it with Everybody. But then you'd have to go into the management console under Shared Folders and choose to stop sharing it, which leaves you with explicit permissions set, as mentioned above.
 
 
 
Another way to get rid of the lock icon is to re-add the permissions that were lost:
 
* add Users and leave the permissions at their defaults
 
* add Authenticated Users and check the Modify permission (which will also enable Write permission)
 
However, as mentioned, this is not the same as resetting the folder to only use inherited permissions.
 
 
 
== Can't rename or delete folders? ==
 
 
 
Windows 7 generates hidden Thumbs.db files in folders with thumbnail-able content, but the OS doesn't actually need them; they're only there for compatibility with older Windows OSes and apps that use those files. In Windows Vista and Windows 7, there's some kind of Explorer bug which sometimes holds these files open, preventing renaming or deletion of their parent folder. It seems to mostly affect video files.
 
 
 
One workaround is this:
 
* Temporarily change the folder view to one of the following: content, medium icons, large icons, extra-large icons.
 
 
 
Another workaround is to disable Thumbs.db generation with a group policy update. There are various instructions around for doing this, but this is the simplest (do it as an Administrator, and remember you have to do Run as Administrator when opening the command shell):
 
* <code>REG ADD "HKCU\Software\Policies\Microsoft\Windows\Explorer" /v "DisableThumbsDBOnNetworkFolders" /t REG_DWORD /d 1 /f</code>
 
 
 
You can then check the status of the key:
 
* <code>REG QUERY "HKCU\Software\Policies\Microsoft\Windows\Explorer" /v "DisableThumbsDBOnNetworkFolders"
 
 
 
After disabling Thumbs.db generation, you might want to just go ahead and delete all your existing Thumbs.db files (do it from the root of each drive):
 
* <code>del /F /S /A:H Thumbs.db</code>
 
 
 
Here are a couple of discussion threads related to this:
 
* http://social.technet.microsoft.com/Forums/en-US/w7itproui/thread/c0817162-3fcd-4823-bdf8-70b2936cd51b#5f1ebc10-dd44-400e-a356-d585d7a6c851
 
* http://social.technet.microsoft.com/Forums/en-US/w7itprogeneral/thread/ca2cbc1a-362f-4f01-a8f8-6f05112f1915/
 
 
 
One of the posts mentions the possibility that it has something to do with SMB2 Client Redirector cache, but no one seems to have checked into this:
 
* http://technet.microsoft.com/en-us/library/ff686200(WS.10).aspx
 
 
 
== Remote Desktop configuration ==
 
 
 
This can't be done on Windows 7 Starter.
 
 
 
# Go to Remote Settings (it's in the System Properties, e.g. right-click on My Computer and go to Properties).
 
# Choose one of the "Allow connections..." settings for Remote Desktop. If you choose Network Level Authentication, it will probably lock out non-Win7 clients.
 
# Click Apply or OK.
 
 
 
That's all you need to get it going, but for extra security, I suggest changing the listening port. Details are in [http://support.microsoft.com/kb/306759 a Microsoft Knowledge Base article], but basically you just run Regedit, go to HKEY_LOCAL_MACHINE\System\CurrentControlSet\Control\TerminalServer\WinStations\RDP-Tcp\PortNumber and enter the port number. The change will take effect after the next reboot. Don't forget to create a custom rule in Windows Firewall to allow TCP traffic inbound on that port. You can disable all the other Remote Desktop rules; they are for the default port. Of course, you will need to make sure that you include the port number after the computer hostname in the client's logon settings, like ''the.remote.host:12345''.
 
 
 
I also suggest setting an account lockout threshold in the Local Group Policy Editor > Local Computer Policy > Computer Configuration > Windows Settings > Security Settings > Account Policies > Account Lockout Policy. When you enter a threshold, it will suggest 30 minutes for the other values; this is good. This will make brute-force attacks difficult. I set mine to 5, which should be enough retries for a real person who just can't remember or is fat-fingering their password.
 
 
 
== Application compatibility mode ==
 
 
 
To set Windows XP Service Pack 3 mode for all .exe files in a folder:
 
 
 
:<code>for %x in ("*.exe") do reg.exe Add "HKCU\Software\Microsoft\Windows NT\CurrentVersion\AppCompatFlags\Layers" /v "%~fx" /d "WINXPSP3"</code>
 
 
 
If you want it to be for all users, use HKLM instead of HKCU. If you do that, though, you won't be able to change it in the file Properties > Compatibility dialog.
 
 
 
The <code>/d</code> value is a space-separated list of values. The first value must be one of:
 
* <code>WIN95</code>
 
* <code>WIN98</code>
 
* <code>WIN4SP5</code>
 
* <code>WIN2000</code>
 
* <code>WINXPSP2</code>
 
* <code>WINXPSP3</code>
 
* <code>VISTARTM</code>
 
* <code>VISTASP1</code>
 
* <code>VISTASP2</code>
 
* <code>WIN7RTM</code>
 
* <code>WINSRV03SP1</code>
 
* <code>WINSRV08SP1</code>
 
 
 
The rest are optional:
 
* <code>256COLOR</code>
 
* <code>640X480</code>
 
* <code>DISABLETHEMES</code>
 
* <code>DISABLEDWM</code>
 
* <code>HIGHDPIAWARE</code>
 
* <code>RUNASADMIN</code>
 
 
 
To unset:
 
 
 
:<code>for %x in ("*.exe") do reg.exe Delete "HKCU\Software\Microsoft\Windows NT\CurrentVersion\AppCompatFlags\Layers" /v "%~fx" /f</code>
 
 
 
Leave off the <code>/f</code> if you want to be prompted for confirmation every time.
 
 
 
==Java updates==
 
The 32-bit JRE installation (in <code>C:\Program Files (x86)\Java</code>) updates itself, but produces strange warnings about certificate problems. There's apparently no fix for it.
 
 
 
The 64-bit JRE installation (in <code>C:\Program Files\Java</code>) does '''not''' support auto updating yet (a [http://bugs.sun.com/view_bug.do?bug_id=6492837 known issue] since 2006). So you have to manually install the latest version whenever is convenient.
 
 
 
Here's where to download the latest versions (this week):
 
* http://www.oracle.com/technetwork/java/javase/downloads/jre7-downloads-1880261.html
 
 
 
===Java and Serviio===
 
If a 64-bit Java app like Serviio is running when you do the update, you'll be prompted for a reboot. But even after the reboot, Java doesn't be installed correctly and the Serviio service won't start. You'll get a warning about needing to set EXE4J_JAVA_HOME to point to a 64-bit JRE. Ignore the warning and just re-run the 64-bit JRE installer without Serviio running. It should prompt you about that version of Java already being installed; just continue and it should complete the installation without error. Then you can go start the Serviio service manually.
 
 
 
Next time, before running the 64-bit JRE installer, just right-click on the Serviio icon in the system tray and choose "Exit Serviio" to shut down Serviio Console and the Serviio service. After installing the new JRE, restart both from Start Menu > Serviio.
 
 
 
===Java control panel===
 
If you're running a 64-bit version of Windows, the Java control panel will be the 64-bit version, which doesn't have an Update tab due to the issue mentioned above. I assume this control panel only affects the 64-bit JRE. You can also point it to your 32-bit JRE, but I don't know if doing so has any effect.
 
 
 
= Font related =
 
 
 
* On Discogs we need to figure out the best fonts to use for the [[phonorecord copyright symbol]] "℗".
 
* I'm keeping track of my GDI++ testing [[User:Mjb/GDI++|here]]. GDI++ allows Windows apps to use FreeType2 for font rendering, rather than Windows' built-in GDI libraries.
 
* On Discogs, more and more people are using "N" followed by the degree symbol to approximate a [[numero sign]]. I used to be successful at converting these to "No." but moderators are becoming increasingly argumentative about this not being "as on the release". My position is that there's ''no way'' to use Unicode characters ''alone'' to approximate the kind of formatting found in a proper English numero sign, just as one can't produce 1<sup>st</sup> or certain other superscript/subscript characters.
 
 
 
==Replacing Tahoma and Microsoft Sans Serif==
 
On Windows 2000 and Windows XP, these TTF files are watched by Windows File Protection (WFP):
 
* %systemroot%\fonts\micross.ttf
 
* %systemroot%\fonts\tahoma.ttf
 
* %systemroot%\fonts\tahomabd.ttf
 
 
 
In addition, tahoma.ttf is always "in use" and cannot be deleted. The other two can be deleted, but get replaced by WFP right away.
 
 
 
You can turn off WFP for specific files by following directions at the bottom of Jeremy Collake's [http://www.bitsum.com/aboutwfp.asp Hacking Windows File Protection] page. Specifically, here is what to do:
 
 
 
# Get a hex editor that can search for UTF-16LE text strings, like the freeware [http://www.chmaas.handshake.de/delphi/freeware/xvi32/xvi32.htm XVI32].
 
# Copy %systemroot%\system32\sfcfiles.dll to a temporary location.
 
# Edit it using the hex editor. Search for micross, and change the "m" so that it is hex 00, not 6D. Do this by clicking the "6D" and then typing "00". Be careful not to change the "m" to "0" (hex 30).
 
# Do the same for the two tahomas. NOTE: There are multiple instances of each filename. Change them all!
 
# Save the file and exit.
 
# Extract PEChksum.exe from [http://www.bitsum.com/files/pesuite.zip pesuite.zip] and run it from the command line:<br><code>PEChksum.exe sfcfiles.dll</code>
 
# Extract MoveLatr.exe from [http://www.bitsum.com/files/movel_101.zip movel_101.zip] and run it from the command line:<br><code>MoveLatr.exe sfcfiles.dll %systemroot%\system32\sfcfiles.dll</code> (answer Y when prompted to overwrite).
 
# It seems like it'd be a good idea to do this for the cached copy as well:<br><code>copy sfcfiles.dll sfcfiles2.dll</code><br><code>MoveLatr.exe sfcfiles2.dll %systemroot%\system32\dllcache\sfcfiles.dll</code>
 
# Reboot.
 
# You should now be able to delete and replace micross.ttf and tahomabd.ttf without having them become replaced right away. If you then try to delete tahoma.ttf, you might notice the dialog box telling you it's busy now uses your updated micross.ttf!
 
# Replacing tahoma.ttf requires booting from another partition and OS from which you can access the file and replace it manually, or you could just use MoveLatr.exe again:<br><code>MoveLatr.exe modifiedTahoma.ttf %systemroot%\Fonts\tahoma.ttf</code>
 
 
 
=MP3 related=
 
 
 
==Lossless MP3 editing on Windows==
 
 
 
To be absolutely safe:
 
 
 
# <code>[http://omion.dyndns.org/mp3packer/mp3packer.html mp3packer] -b 320 -r inputfile.mp3 tempfile.mp3</code> to all but eliminate bit reservoir usage
 
# Edit <code>tempfile.mp3</code> in [http://mpesch3.de1.cc/mp3dc.html mp3DirectCut]
 
# <code>mp3packer -s -t -z tempfile.mp3 outputfile.mp3</code> to recompress to VBR
 
 
 
However it's generally safe just use mp3DirectCut on the files, if all you're doing is trimming silence/noise from the ends. Generally. Problems:
 
* If you cut off frames from the beginning, then the new first frame(s) might not be decodable because their main data began up to 511 bytes back, not counting frame headers. That is, it was in the frames that got cut off. It would be nice if mp3DirectCut would color such frames so you can make an informed decision as to which ones are safe to cut, if any.
 
* If you cut off the last frame, then the last 529 samples of the last frame you kept won't be decoded because there's no more padding.
 
* Due to granule overlap, 288 samples at each truncated end will be about inaccurate and too quiet because the rest of their info was in a frame that's no longer there.
 
I'm not sure how mp3DirectCut deals with encoder padding and delay in the LAME tag.
 
 
 
There's another lossless editor called [http://www.mpgedit.org/ mpgedit]. There's a console version (mpgedit) and a graphical version (xmpgedit), and it is available for multiple platforms. However, when I tried it on Windows, it didn't really work at all. Its editing interface isn't intuitive, and mp3DirectCut seems to just be a little better all around.
 
 
 
I used to use [http://www.mptrim.com/ mpTrim] for trimming silence from the ends, but the freeware version only works on smallish files, and it has nothing resembling a wave editor, so you have to 'earball' it. I now prefer mp3DirectCut.
 
 
 
==MP3 splitting==
 
 
 
MP3 '''splitting''', such as with a cue sheet, is a can of worms. MP3 frames always output a fixed number of samples (1152, if 44.1 kHz). Thus, cutting on a frame boundary means cutting on an 1152-sample boundary, usually. You can't really cut with any more precision than that. However if you include an extra frame beyond the one you're cutting in, and then adjust encoder padding & delay values, your "gapless playback"-aware decoder can output just the samples you wanted. AFAIK, the only splitting tool that will do this correctly is called '''[http://www.hydrogenaudio.org/forums/index.php?showtopic=35654 pcutmp3]''', a java command-line program. mp3DirectCut does let you overlay a cue sheet, but it only cuts on frame boundaries, doesn't include overlapping frames, and it messes up the padding & delay tags.
 
 
 
==MP3 gapless playback==
 
 
 
LAME always stores padding & delay values in the LAME tag, so it's always "gapless" in the sense that the number of samples output will be correct, given a decoder that supports those values. But it's my understanding that if you just encoded the tracks one at a time, LAME didn't have access to the previous and next track's frames, which are needed in order to know how to ''best'' encode the frames near the boundaries of the current track. From what I've read, it seems 288 samples on either side of the boundary between a pair of tracks will be at half volume (not confirmed, though). The psychoacoustic model may also be affected for adjacent frames (I think 2 frames before & after). I can't say with certainty what really happens, but you don't get the same samples at the boundaries when encoding separate tracks as when encoding the whole album at once. So the proper thing to do, to get split tracks, is to encode the entire set of tracks ''all at once'' by invoking LAME with the <code>--nogaptags --nogap</code> options. The problem, though, is that the resulting files will have no padding (except in the last file), which is a situation that players (even foobar2000) generally don't account for. These unpadded files won't be played to the very end. What a player has to do, to support playback of such files, is start feeding the next file's frames to the decoder when the end of the current file is reached. As far as I know, very few players do it. Rockbox supposedly does. Do any others?
 
 
 
Anwyay, the HydrogenAudio forums have a number of posts suggesting that <code>--nogap</code> is deprecated or obsolete, but unless I'm mistaken, that's clearly not the case. Abandoned, yes. Deprecated for basic gapless playback, yes. But obsolete, no. If properly supported in players, it should offer superior gapless playback, at least from a technical perspective, if not an audible difference.
 
 
 
I'd be interested to see the results of splitting a pair of tracks with pcutmp3, as compared to encoding them with <code>--nogaptags --nogap</code>. Does it produce the same frames and padding/delay values? Even if it doesn't, does it produce the same output when decoded?
 
 
 
==MP3 SHOUTcast stream transcoding==
 
 
 
Before I switched to AAC+, I was looking into offering a low-bitrate MP3 version of my audio stream by connecting to the regular stream as a client, transcoding it, and feeding it to another server. This isn't necessary when using recent versions of the SHOUTcast DSP for Winamp; it can feed multiple streams at different bitrates to different servers. But I was using an older DSP so that I could use an external MP3 codec, and had to come up with a workaround. I didn't get very far in my research, but did find several options for grabbing a stream to a file on Unix:
 
 
 
mpg123 -b 4096 -s http://212.72.165.24:9154/ > out.mp3
 
gnetcat -l -p 8081 212.72.165.24 9154 | lame --mp3input -b 64 - out.mp3
 
mpg123 -b 4096 -s http://212.72.165.24:9154/ | lame --mp3input -b 64 - out.mp3
 
 
 
And then there was also [http://ficy.sourceforge.net/ fIcy]: an icecast/shoutcast stream grabber suite.
 
 
 
I don't remember if I tried any of these. The idea though was that if I could send the data to a file, the file could actually be a named pipe into icecast or something. However it didn't occur to me that I'd probably lose the protocol data (stream info and song titles). So it was starting to look like the only real option was going to be [http://www.oddsock.org/tools/streamTranscoder/ streamTranscoder], last updated in 2004.
 
 
 
I don't care anymore. I'm just keeping this info for possible future reference, because I remember it was hard to find some of this info.
 
 
 
==Info about my old SHOUTcast setup==
 
My previous setup used the 'Radium' pirated Fraunhofer MP3 codec at 96 kbps CBR, which provided excellent sound quality due to its lowpass filter discarding frequencies above 11.5 kHz, and its ability to force the use of L/R stereo mode rather than poorly-tuned M/S. MP3 codecs, especially Fraunhofer's, do very well at sub-128 kbps bitrates, especially when their lowpass filters are set to cut off the bandwidth-intensive higher-frequency audio.
 
 
 
Why didn't I use LAME? Well, I tried it several times, but for various reasons, I was required to use a codec packaged as Windows .acm file, and LAME's ACM build isn't nearly as tunable as the command-line or DLL versions. The biggest problem is that they've set its lowpass filter a bit too high, around 14 kHz, I think, with no way to change it. The result is inaccurate, noisy upper midrange, which is tolerable for speech, but that's too much high end to try to cram into a 96 kbps MP3 stream of music; something's got to give. They try to make up for it a bit by requiring joint stereo, which lets the encoder decide whether to use L/R or M/S in each frame. A lot of effort has gone into tuning LAME at higher bitrates so that it makes a good guess for that decision, but at 96 kbps CBR, it's no good; the result is too much noise and/or poor stereo separation. If I could do the encoding with LAME's command-line interface, it would be no problem, because I could just force L/R stereo mode and a ~12.5 kHz lowpass, which give it better high end than the Radium codec, with no degradation. If the stream wasn't limited to CBR, I'd use VBR and joint stereo, for which LAME has been tuned very well.
 
 
 
I use AAC+ now. Screw MP3.
 
 
 
==Custom compiling AACGain==
 
I'm going to try to build an [[:wikipedia:Streaming SIMD Extensions|SSE]]-optimized (Pentium III and up) version of [http://altosdesign.com/aacgain/ AACGain]. I'll trace my steps [[User:Mjb/AACGain|here]].
 
 
 
= Video related =
 
 
 
== DVD format basics ==
 
Here's what I've learned:
 
* VIDEO_TS folder contains everything for video DVDs. AUDIO_TS is usually empty.
 
* TS means title set.
 
* IFO = DVD container for various types of navigation info, including title & chapter locations; basically it has everything except the audio/video/image/graphics data.
 
** VIDEO_TS.IFO = main IFO for the whole disc. Some apps can read this and go get the individual title set IFOs.
 
** VTS_01_0.IFO = 1st IFO for title set 01. There can be more. Some apps have to be explicitly pointed to each one of these.
 
** A title is a logical division of the content. Each title is divided into 1 or more chapters. Titles and chapters are numbered 01, 02, 03, etc., and are defined by the timecode of where they begin.
 
** BUP = backup of IFO file.
 
* VOB = DVD container for data. Usually there are as few VOB files as possible, max 1 GB each.
 
* A VOB contains an MPEG-PS (MPEG Program Stream), which can be more flexible, but for DVDs, is a multiplexed combo of the following:
 
** 1 4:3 or 16:9 video stream: '''Usually MPEG-2''', max 9800 Kbps; 24 fps progressive or 29.97 fps interlaced 720x480, 704x480, 352x480 (NTSC, displayed @ 29.97 fps) or 720x576, 704x576, 352x576 (PAL/SECAM, 25 fps interlaced). Sometimes MPEG-1, max 1856 kbps; 352x240 (NTSC) or 352x288 (PAL/SECAM). Sometimes MPEG-4 (DivX/XviD).
 
** 0-8 MPEG audio streams: '''Usually MP2''' (48 KHz, max 384 Kbps). Sometimes MP1, MP3, or AAC. I found that MP3 & AAC aren't always supported!
 
** 1 private stream, with multiplexed sub-streams:
 
*** 0-32 subpictures (subtitle & menu bitmap graphics)
 
*** 0-8 LPCM audio streams
 
*** 0-8 DTS audio streams
 
*** 0-8 SDDS audio streams
 
* A VOB stream is divided into "packs", each 2 KB (one DVD sector).
 
** There are 4 types: video, audio, subpicture, navigation.
 
** Each contains a header and a set of packets for the relevant type of stream. The packet format is complicated.
 
* The file name extension .VOB is often synonymous with .MPG or .MPEG.
 
* '''VOB files can be concatenated.'''
 
 
 
==Splitting DVDs==
 
I often want to losslessly split custom compilation DVDs of concerts and music videos into separate VOB files for each song. The goal is to just segment the various content data streams from the original VOBs and put each segment into its own VOB, without re-encoding the stream. Lots of splitters and multipurpose editors exist, but they are not lossless; they re-encode the streams.
 
 
 
PgcDemux and MPEG Video Wizard are Windows apps that can do lossless splitting. PgcDemux is free but crude and unmaintained. MPEG Video Wizard is more complicated and is not free, but it is more robust and is supported.
 
 
 
===PgcDemux===
 
PgcDemux can be used from the command line, but the rudimentary GUI lets you poke around and see how things are split up. DVDs can be authored in different ways; the separate videos may not actually be indexed separately. Even if they are, there are different indexing methods, which can require different extraction methods. Sometimes the clips just don't line up with VID or cell boundaries. In that case, PgcDemux is useless. But if they do, it's good.
 
 
 
# Make sure all Options are unselected, except "Create a PGC VOB" and "One file per VID".
 
# In VOB id Selection, select "Titles".
 
# Drag and drop the VTS_01_0.IFO into PgcDemux
 
# Select mode "by VOB id" and see if the VOB Id Selection shows a list of VIDs (1 for each video).
 
:: If there's 1 VID for each video, then select mode "by PGC"; the list should show one long PGC and Cells should be the # of videos. You can then click "Process!" and you'll get one VOB file per video (you'll be prompted for where to put them).
 
:: If the VOB Id Selection only shows one long VID, same as the PGC, then you have to select mode "Single Cell". Now the list should show each video. When you click "Process!" you'll only get the currently selected video. The filename will be the same every time, so you have to go rename the file, select the next one, click Process, over and over.
 
:: There will be another problem: some videos may be split into multiple cells. It's OK, just extract all the cells and then use VOBMerge to put them back together into a single VOB file. Another option would be to load the IFO into VIDChanger, select "Fix Cell_Id" and have it write out a new set of VOBs + an IFO which can then be used in "by PGC" mode in PgcDemux.
 
 
 
* PgcDemux & VIDChanger: &lt;[http://download.videohelp.com/jsoto/dvdtools.htm http://download.videohelp.com/jsoto/dvdtools.htm]>
 
* VOBMerge: &lt;[http://www.evilmaster.net/index.php?page=tools http://www.evilmaster.net/index.php?page=tools]>
 
 
 
===MPEG Video Wizard===
 
MPEG Video Wizard is non-free software for DVD-oriented video editing.
 
 
 
====Easy method====
 
The main way to split is very easy: Open the DVD Reader tool, point to the VIDEO_TS folder, then select all the chapters, right-click and choose Extract to VOB. Each chapter will go into its own VOB. That's it! Obviously, this method is useless if the chapter boundaries don't match up with the video boundaries.
 
 
 
====DIY method====
 
If the chapter boundaries aren't where you want to split, then you need to tell it where the boundaries are. Here is the method I use:
 
 
 
Use the DVD Reader, point to the VIDEO_TS folder, select the title(s) you want, and click on Add To Timeline, then close the DVD Reader. In the Time Line window, there will be markers (red dots) for the chapters. Clear them by pressing "Alt"+"M" (or click the X'd-out clover/club button). You want to add your own markers. Start by pressing "M" (or click the clover/club button) to create a marker at the start of the file, to indicate this is where the first chapter begins.
 
 
 
You want to create a marker where each chapter is to begin, so make sure the frame you want to be the first one in the chapter is showing in the Output window. To navigate, on the ruler, drag the white triangle, and use the left and right arrow keys for frame-by-frame movement. Use the plus/minus buttons to zoom in on the timeline. There's a horizontal scrollbar at the bottom. Use "M" to toggle a marker. You can use "Ctrl"+"Tab" or "Ctrl"+"Shift"+"Tab" to navigate from marker to marker (or use buttons on the lower right). "Ctrl"+"Z" will undo. The little disk icon will save your progress in a project file. To help find the right places to edit, you can double-click the video (blue bar) in the timeline, and the preview should appear in the Input window, along with a Bookmarks window which you can close; you can play the video and navigate in the Input window to find the frames you want, then navigate in the Timeline and/or Output Window to the same frame. If you play or navigate in the Output window, it adjusts the position in the Timeline, too, so that's another option for you.
 
  
Once all the markers are in place, don't use Export ("E" or the red-circle button). Instead, click the "DVD" button to open the DVD Editor. It will have a chapter for each marker. Test by clicking on each chapter and playing (yet another way to navigate!) If anything's wrong, go back and make adjustments in the timeline. When you re-open or just click in the DVD Editor window again, the adjustments will be there. Make sure that the V and A bars at the bottom of the window are solid blue, indicating Stream Copy. If they are red, something is wrong. When you're ready, click Export, choose a destination folder, and it will write a VIDEO_TS folder there. Now you can use the DVD Reader tool to do the split, as described in the main method above.
+
=== MP3 & AAC ===
 +
* In 2008–2010, I made some notes about [[User:Mjb/MP3|MP3]] editing, splitting, and gapless playback.
 +
* In 2008, I experimented with [[User:Mjb/SHOUTcast|SHOUTcast]] stream transcoding.
 +
* In 2008, I used Visual C++ 2005 for the first time, and logged my experience compiling [[User:Mjb/AACGain|AACGain]] with it.
  
* MPEG Video Wizard: &lt;[http://www.womble.com/ http://www.womble.com/]>
+
=== DVD ===
 +
* In 2012, I made notes regarding [[User:Mjb/DVD|DVD]] file formats and splitting DVD content into custom files.

Revision as of 02:26, 27 March 2013

Topics

Music & audio

  • In 2008–2010, I tried to keep track of DJ Beppe Loda mixes and links.
  • In 2013, I made some notes regarding RTLSDR, which is software-defined radio for a receiver based on the RTL chip.

XSLT

Windows 7

Fonts

MP3 & AAC

  • In 2008–2010, I made some notes about MP3 editing, splitting, and gapless playback.
  • In 2008, I experimented with SHOUTcast stream transcoding.
  • In 2008, I used Visual C++ 2005 for the first time, and logged my experience compiling AACGain with it.

DVD

  • In 2012, I made notes regarding DVD file formats and splitting DVD content into custom files.