Header image San Francisco, California
USA

xEmP - Lightroom Plugin

Saves xmp sidecar files for changed photos.

Supports sidecars for all file formats, including virtual copies.

Featuring:

System Requirements

Quick Links (intra-page)

Background Information
Screenshots
How to Use
xEmP FAQ (Frequently Asked Questions)
Revision History
Download

See the readme file after downloading for usage and other notes.

 


xEmP - Screenshots

 

Plugin Manager

 

File Menu -> Plugin Extras

 

Background/Introduction

Inspired by the desire to be able to save develop and metadata edits in smaller xmp files for backup purposes, and to not lose virtual copies when falling back to an earlier catalog, and to have all photo info in xml format for consistent handling and/or further analysis and processing.

 

Definitions (for the purposes of xEmP)

xmp Develop settings and metadata, normally embedded in DNG files, but can also be kept in a sidecar. Note: Bridge/ACR/Photoshop will write xmp sidecar if DNG master is read-only, but Lightroom won't. Lightroom will however read xmp sidecars of DNG when present, if newer than embedded xmp.
xmp sidecar xmp in a separate xml file alongside the source photo file. Sidecars conforming to xmp spec have xmp extension. DNG & proprietary raw formats both support xmp sidecars.
rgb sidecar sidecars for RGB photos contain all the same information as the xmp block in the source files. However the format is slightly different, and Lightroom can't (won't) read them anyway, so they are given an xml extension, to distinguish. ExifTool and/or xEmP required for restoral.
virtual copy sidecar virtual copy metadata may be stored in an xml or lua text file - its contents are not xmp compatible. ExifTool can not restore this metadata, xEmP (or equivalent) must do it.
target in this context, it usually means a photo subject to saving/reading xmp metadata. Note: if no photo is selected, then all photos in the filmstrip are targeted.
DNG A file format that can store raw or rgb image data, and metadata. Despite being a raw container, it is considered a different file format than RAW to plugins - hopefully distinctions will be clear in context.
RAW Sometimes means raw images regardless of file format, and sometimes is synonymous with "proprietary raw" file format. I try to reserve upper case to mean the file format, and lower case to mean a raw image independent of container.
RGB sometimes means non-raw image data, but sometimes means TIFF, JPG, or PSD file format. Sorry if this is confusing...
   

 

How to Use the xEmP

- Install (see readme file in downloaded zip file)
- Configure in plugin manager
- File Menu

 

Plugin Manager Configuration

See elare plugin framework for common settings.

Additional Settings and Controls

xEmP Settings
Compare new xmp/xml data to existing file contents before saving sidecar

Double checks that contents to be written are different than current xmp sidecar file contents before writing xmp file.

Its more efficient and recommended to leave this box unchecked, unless you suspect change detection may be wonky, then try enabling this for a while.

Note: If this setting is enabled, last-edit-time of photo will *always* be compared to xmp file time, and if there is any discrepancy between the result of comparing data byte-by-byte and the normal change detection mechanism, a warning will be issued. So enabling this option is a good way to gain confidence in the normal change detection mechanism, although its considerably slower ;-}

Assume Lightroom metadata is already saved

Check this box if you work with the Lightroom preference "auto-save XMP" enabled, or if you plan to always save Lightroom metadata manually before invoking xEmP save functions. If you keep finished files locked using ChangeManager, then check this box.

Save virtual copies as xml Check this box for xml format. The files will be bigger, but they'll be xml like your other xmp or pseudo-xmp files (recommended). Uncheck for lua text format file if you prefer, or if there are any problems encoding/decoding metadata, in which case please let me know...
Save rgb and virtual copy metadata in catalog directory Check this box to keep rgb and virtual copy metadata files in catalog folder instead of source photo folders. This is strongly recommended *iff* all your photo filenames are unique, in which case, moving photos and virtual copies to different folders will not result in dissociation. If you allow duplicate filenames in your catalog, then uncheck this box. Or, if you just like having them as sidecars in source folder, and you never move photos anyway..., then no good reason not to uncheck it...

 

File Menu

Access via Lightroom's menu bar: File Menu -> Plugin Extras

Save xmp for most selected photo Determine if source photo has changed, and if so, save xmp sidecar corresponding to most selected photo.
Save xmp for target photos Target photos are those selected, unless none are selected, in which case its the entire filmstrip.
Save xmp for whole catalog Note: photos buried in stack may not be updated, since metadata is only saved in Lightroom for photos that are selected, and photos can only be selected if they are at the top of a collapsed stack, or are not in a collapsed stack.
Read xmp for most selected photo Touches xmp sidecar file of most selected photo, then if Windows OS invokes Lightroom's read metadata function which will then read the sidecar. On Mac you have to initiate the metadata read yourself after invoking this function.
Read xmp for target photos. Targets selected photos or entire filmstrip
Read xmp for whole catalog Targets all photos not buried in stack.
Collect changed photos Creates a collection of photos that have been edited since their xmp sidecars were last saved. Note: this function uses a different targeting scheme than the others - number of photos being taken into consideration is displayed in the prompt. In a nutshell, if zero or one photo is selected, the whole filmstrip is targeted, otherwise the selected photos.
Recreate missing virtual copies from sidecars If a virtual copy already exists in the catalog, one should use the "read xmp..." functions to restore saved metadata. If however the virtual copy has been deleted, or you are recreating a catalog from saved xmp/xml metadata, this function can be used to recreate all virtual copies. Limitations apply (see below).
Cleanup orphaned sidecars

Two cases:

1. You keep rgb and virtual-copy metadata as sidecars in photo source directories, in which case: If you move rgb or (raws with) virtual-copy photos with sidecars to another folder (in Lightroom), Lightroom will not recognize the "unconventional" sidecars, thus they will become "orphaned". I could have coded a way to reassociate orphaned sidecars with their original, but decided to just allow them to be moved to trash instead. - If you move rgb photos or raws+virtual-copies to another folder, I recommend re-saving the sidecars there, then cleanup orphaned sidecars from previous folder(s) - if not immediately, as part of periodic maintenance, unless you don't mind the extraneous files hanging around. Also sidecars remain after removing virtual copies in Lightroom - this function will clean 'em up.

2. You keep rgb and virtual-copy metadata in catalog folder (recommended), in which case extraneous "xmp" files can only result from virtual copy deletion.

 

 


 

xEmP FAQ (Frequently Asked Questions)

(no particular order)


These FAQs come partly from users, and partly from my imagination. Please let me know if there are errors or omissions in this FAQ - thanks.

NOTE: The following Q&A's assume that the plugin is working as I expect... If, after your best effort, still "no go", please let me know.


Question: Why would I ever need or want such a thing as xEmP?

Answer: This plugin is for people who like the security, convenience, consistent handling, and interoperability afforded by having a separate, smaller, human and computer readable xml text file (along with their big binary source photo files), for all file-types, which only changes when the corresponding photo changes, for expedient backup, potential restoral, and whatever else...


Question: What's the difference between Lightroom's save metadata function and xEmP's?

Answer: Depends on format:

DNG - Lightroom saves to source file, not sidecar. xEmP extracts from source file and saves to xmp-spec compliant sidecar, which Lightroom can read, and will read under the right circumstances (see below).

RGB (TIFF, JPG, PSD) - Lightroom saves to source file, not sidecar. xEmP extracts from source file and saves to exif-tool xml file.

RAW (proprietary non-DNG) - xEmP just invokes Lightroom's save metadata function, but can deselect unchanged photos first, so sidecars of unchanged raw photos are not re-written.

VIRTUAL - Lightroom will not save xmp metadata for virtual copies, nor will exif-tool. xEmP saves (develop settings and) metadata in xml or lua text file.

All formats - Lightroom saves metadata for all selected files whether its changed or not. xEmP will not save xmp if photo has not been edited since last save (unless you choose). Also, xEmP allows you to specify target options without necessarily changing your present selection

VIDEO not supported.


Question: What's the difference between Lightroom's read metadata function and xEmP's?

Answer: Depends on format:

DNG: Lightroom reads from sidecar or source file based on last-modified-date - whichever is more recent is read. xEmP touches the DNG's xmp sidecar (changes last-modified date) so Lightroom will choose it when reading metadata. xEmP will invoke Lightroom's read-metadata function after touching.

RGB (TIFF, JPG, PSD): Lightroom reads metadata from source file. xEmP uses exiftool to copy saved metadata to source file then invokes Lightroom's read-metadata function.

RAW: No difference.

VIRTUAL: Lightroom does not support reading virtual copy metadata. xEmP reads saved metadata from sidecar file and applies it to virtual photos using functions of the SDK.

All formats: Lightroom always targets selected photos whereas xEmP's read functions allow targeting options, and restore your present selection after reading.


Question: I saved xmp metadata using xEmP but it does not seem to be up to date - what happened?

Answer: Most likely xmp metadata was not saved to source files first (DNG & RGB). xEmP's sidecar metadata is only as fresh as the source from which it was extracted. Another possibility is faulty change detection - consider enabling "Compare new xmp/xml data to existing file contents before saving sidecar option" for a while. The other possibility is that there is a bug in xEmP or ExifTool or Lightroom... - please let me know if there are any problems.


Question: What is the difference in xmp sidecars for DNG extracted by exiftool and xmp sidecars for DNG written by Bridge/Photoshop/ACR?

Answer: The format is slightly different but the contents are the same. These xmp sidecar files should be interchangeable. If they are not, then please let me know so I can let Phil Harvey (author of ExifTool) and/or Adobe know about it.


Question: Why don't you use folder batch mode for faster extraction when doing all or most files in a folder?

Answer: I did for a while but suprisingly it isn't that much faster, and the format of xml is different in batch mode vs. non-batch mode, so change detection based on xmp-data is foiled when using different extraction modes. (both modes need to be used, since batch mode is much slower when not doing all files in a folder).


Question: Why are the sidecars for RGB files named differently than the sidecars for DNG files?

Answer: The xmp spec supports sidecars for DNG, just like other RAW files: same base name with 'xmp' extension. However, the same is not true for RGB files. RGB files include the original extension to avoid ambiguity, and use xml for the extension instead of xmp, to indicate that theses files are not xmp-spec compliant - they have the same contents, but Lightroom will not read RGB sidecars - exiftool/xEmP is required.


Question: I said "Save Regardless" but not all sidecars were updated - why not?

Answer: If 'Assume Lightroom metadata is already saved' is checked, sidecars for raws will not be re-saved, even if "Save Regardless" is chosen. Likewise, if 'Compare new xmp/xml data to existing file contents...' is checked files that would not be changed are not resaved. If neither of these two boxes is checked, then sidecars for all selected files should be re-saved when 'Save Regardless' is chosen.


Question: How do you accomplish change detection for proprietary raw metadata saves?

Answer: I use the last-edit-time metadata item and compare to xmp-file modification date to determine whether a photo really needs updating. If not changed, it is deselected before saving Lightroom metadata.


Question: Any other hot tips I should know about?

Answer: Yes:


Question: What are the limitations of xEmP and what are your plans for the future?

Answer:


 

xEmP Revision History

(reverse chronological order)

 

Version 1.8, released 2013-09-04

- Some bug fixes..
- New ExifTool included, for Mac too now (prior versions required Mac users to download & install ExifTool and configure in plugin manager).
- Certified for Lr5.

 

Version 1.7, released 2011-04-30

- Certified for Lr4.

 

Version 1.6, released 2011-11-10

- Added option to save rgb and virtual copy metadata in catalog folder.

- Added special handling for files locked by change manager.
- Re-worked case when 'Assume Lightroom metadata is saved' is set.
- Re-wrote the xml parser / serializer - should be backward compatible, but if files are re-saved once when not changed, there may some minor format issue. As long as they are not re-saved a second time, it should be ok.
- Changed statistics for raw saves, only counts a raw save if xmp file actually changed (it used to be counted as saved if "assume lr metadata already saved" was checked, even though the xmp file was not altered.
- Added checks for missing files to avoid misleading error messages.

###1 - Not tested on Mac yet, but should be compatible.

 

Version 1.5, released 2011-10-25

- Added option to save virtual copy metadata as xml (lua text file used to be only option).

 

Version 1.4, released 2011-10-23

- Added cleanup function to purge extraneous orphaned sidecars that can result when moving photos to a different folder.

 

Version 1.3, released 2011-10-22

- Finalized support for virtual copies - now enabled by default.

 

Version 1.2, released 2011-10-19

- Added (limited/experimental) support for virtual copies. Hint: edit "advanced settings" to enable.

 

Version 1.1, released 2011-10-18

- Added change detection support for non-DNG raw saves.
- Added ability to assemble a collection of photos that have changed since xmp sidecars last saved.

 

Version 1.0, released 2011-10-17

- Initial release - enjoy!

 

Version 0.1, released 2011-10-16

- Experimental status for now - I want to make sure there are no problems with the xmp generated by ExifTool before releasing v1.0 - please let me know if problems or not - thanks.

 


 

Please (IDENTIFY THE PLUGIN) let me know what you think, and please (IDENTIFY THE PLUGIN) report bugs.

 

Download

acceptance of Download Terms & Conditions will be required

xEmP 1.8 - Latest & greatest - this is the one to download.

xEmP 1.6 - A previous version, in case of trouble with latest (please let me know about it).

 

Static content updated 2011-10-16 Copyright 2007 - robcole.com - all rights reserved. Dynamic content updated 04:07:21 AM