Merge pull request #178 from BzKevin/BzKevin-AutoGrouper

Add AutoGrouper

README.md

@@ -1,10 +1,186 @@
-lua-scripts
-===========
+# Lua scripts
 
-The Lua scripts in this repository are meant to be used together with darktable. Either copy them individually to `~/.config/darktable/lua` (you might have to create that folder) or just copy/symlink the whole repository there. That allows to update all your scripts with a simple call to `git pull`.
+## Description
 
-To enable one of the scripts you have to add a line like `require "official/hello_world"` to your `~/.config/darktable/luarc` file which will enable the example script in `official/hello_world.lua` (note the lack of the `.lua` suffix).
+darktable can be customized and extended using the Lua programming language. This repository contains the collected
+efforts of the darktable developers, maintainers, contributors and community. The following sections list the scripts
+contained in the repository, whether they can be run by themselves (Standalone - Yes) or depend on other
+scripts (Standalone - No), what operating systems they are known to work on (L - Linux, M - MacOS, W - Windows), and their purpose.
+
+### Official Scripts
+
+These scripts are written primarily by the darktable developers and maintained by the authors and/or repository maintainers. They are located in the official/ directory.
+
+Name|Standalone|OS   |Purpose
+----|:--------:|:---:|-------
+check_for_updates|Yes|LMW|Check for updates to darktable
+copy_paste_metadata|Yes|LMW|Copy and paste metadata, tags, ratings, and color labels between images
+delete_long_tags|Yes|LMW|Delete all tags longer than a specified length
+delete_unused_tags|Yes|LMW|Delete tags that have no associated images
+enfuse|No|L|Exposure blend several images (HDR)
+generate_image_txt|No|L|Generate txt sidecar files to be overlaid on zoomed images
+image_path_in_ui|Yes|LMW|Plugin to display selected image path
+import_filter_manager|Yes|LMW|Manager for import filters
+import_filters|No|LMW|Two import filters for use with import_filter_manager
+save_selection|Yes|LMW|Provide save and restore from multiple selection buffers
+selection_to_pdf|No|L|Generate a PDF file from the selected images
+
+
+### Contributed Scripts
+
+These scripts are contributed by users. They are meant to have an "owner", i.e. the author, who maintains them. Over time the community has helped maintain these scripts, as well as the authors. They are located in the contrib/ directory.
+
+Name|Standalone|OS   |Purpose
+----|:--------:|:---:|-------
+autostyle|Yes|LMW|Automatically apply styles on import
+clear_GPS|Yes|LMW|Reset GPS information for selected images
+copy_attach_detach_tags|Yes|LMW|Copy and paste tags from/to images
+cr2hdr|Yes|L|Process image created with Magic Lantern Dual ISO
+enfuseAdvanced|No|LMW|Merge multiple images into Dynamic Range Increase (DRI) or Depth From Focus (DFF) images
+fujifilm_ratings|No|LM|Support importing Fujifilm ratings
+geoJSON_export|No|L|Create a geo JSON script with thumbnails for use in ...
+geoToolbox|No|LMW|A toolbox of geo functions
+gimp|No|LMW|Open an image in GIMP for editing and return the result
+gpx_export|No|LMW|Export a GPX track file from selected images GPS data
+hugin|No|LMW|Combine selected images into a panorama and return the result
+image_stack|No|LMW|Combine a stack of images to remove noise or transient objects
+kml_export|No|L|Export photos with a KML file for usage in Google Earth
+LabelsToTags|Yes|LMW|Apply tags based on color labels and ratings
+passport_guide|Yes|LMW|Add passport cropping guide to darkroom crop tool
+pdf_slideshow|No|LM|Export images to a PDF slideshow
+quicktag|Yes|LMW|Create shortcuts for quickly applying tags
+rate_group|Yes|LMW|Apply or remove a star rating from grouped images
+rename-tags|Yes|LMW|Change a tag name
+select_untagged|Yes|LMW|Enable selection of untagged images
+slideshowMusic|No|L|Play music during a slideshow
+video_mencoder|No|L|Export video from darktable
+
+### Example Scripts
+
+These scripts provide examples of how to use specific portions of the API. They run, but are meant for demonstration purposes only. They are located in the examples/ directory.
+
+Name|Standalone|OS   |Purpose
+----|:--------:|:---:|-------
+api_version|Yes|LMW|Print the current API version
+gettextExample|Yes|LM|How to use translation
+hello_world|Yes|LMW|Prints hello world when darktable starts
+moduleExample|Yes|LMW|How to create a lighttable module
+multi_os|No|LMW|How to create a cross platform script that calls an external executable
+preferenceExamples|Yes|LMW|How to use preferences in a script
+printExamples|Yes|LMW|How to use various print functions from a script
+running_os|Yes|LMW|Print out the running operating system
+
+### Tools
+
+Tool scripts perform functions relating to the repository, such as generating documentation. They are located in the tools/ directory.
+
+Name|Standalone|OS   |Purpose
+----|:--------:|:---:|-------
+executable_manager|Yes|LMW|Manage the external executables used by the lua scripts
+gen_i18n_mo|No|LMW|Generate compiled translation files (.mo) from source files (.po)
+get_lib_manpages|No|LM|Retrieve the library documentation and output it in man page and PDF format
+get_libdoc|No|LMW|Retrieve the library documentation and output it as text
+script_manager|No|LMW|Manage (install, update, enable, disable) the lua scripts
+
+## Download and Install
+
+The recommended method of installation is using git to clone the repository. This ensures that all dependencies on other scripts
+are met as well as providing an easy update path. Single scripts listed as standalone may be downloaded and installed by themselves.
+
+### snap packages
+
+The snap version of darktable comes with lua included starting with version 2.4.3snap2. It is currently in the edge channel, but should reach the stable channel soon.
+
+Ensure git is installed on your system. If it isn't, use the package manager to install it. Then open a terminal and:
+
+    cd ~/snap/darktable/current
+    git clone https://github.com/darktable-org/lua-scripts.git lua
+
+### flatpak and appimage packages
+
+These packages run in their own environment and don't have access to a lua interpreter, therefore the scripts can't run. The packagers could enable the internal interpreter, or allow the package to link the interpreter from the operating system, or bundle a copy of lua with the package. If you use one of these packages and wish to use the lua scripts, please contact the package maintainer and suggest the above fixes.
+
+### Linux and MacOS
+
+Ensure git is installed on your system. If it isn't, use the package manager to install it. Then open a terminal and:
+
+    cd ~/.config/darktable/
+    git clone https://github.com/darktable-org/lua-scripts.git lua
+
+### Windows
+
+Ensure git is installed on your system. Git can be obtained from https://gitforwindows.org/, as well as other places. If you use the gitforwindows.org distribution, install the Git Bash Shell also as it will aid in debugging the scripts if necessary. Then open a command prompt and run:
+
+Open a command prompt.
+
+    cd %LOCALAPPDATA%\darktable
+    git clone https://github.com/darktable-org/lua-scripts.git lua
+
+## Enabling
+
+When darktable starts it looks for a file name `~/.config/darktable/luarc` (`%LOCALAPPDATA%\darktable\luarc` for windows) and reads it to see which scripts to include. The file is a plain text file with entries of the form `require "<directory>/<name>"` where directory is the directory containing the scripts, from the above list, and name is the name from the above list. To include GIMP the line would be `require "contrib/gimp"`.
+
+You can also create or add lines to the luarc file from the command line:
+
+`echo 'require "contrib/gimp"' > ~/.config/darktable/luarc` to create the file with a gimp entry\
+or `echo 'require "contrib/hugin"' >> ~/.config/darktable/luarc` to add an entry for hugin.
+
+On windows from a command prompt:
+
+`echo 'require "contrib/gimp"' > %LOCALAPPDATA%\darktable\luarc` to create the file with a gimp entry\
+or `echo 'require "contrib/hugin"' >> %LOCALAPPDATA%\darktable\luarc` to add an entry for hugin.
+
+## Disabling
+
+To disable a script open the luarc file in your text editor and insert `--` at the start of the line containing the script you wish to disable, then save the file.
+
+## Updating
+
+To update the script repository, open a terminal or command prompt and do the following:
+
+### Snap
+
+    cd ~/snap/darktable/current/lua
+    git pull
+
+### Linux and MacOS
+
+    cd ~/.config/darktable/lua/
+    git pull
+
+### Windows
+
+    cd %LOCALAPPDATA%\darktable\lua
+    git pull
+
+## Documentation
 
 Each script includes its own documentation and usage in its header, please refer to them.
 
-In order to have your own scripts added here they have to be under a free license (GPL2+ will definitely work, others can be discussed). Scripts in the `official/` subfolder are maintained by the darktable community, those under `contrib/` are meant to have an "owner" who maintains them.
+Lua-script libraries documentation may be generated using the tools in the tools/ directory.
+
+More information about the scripting with Lua can be found in the darktable user manual:
+https://www.darktable.org/usermanual/en/lua_chapter.html
+
+The darktable Lua API documentation is here:
+https://www.darktable.org/lua-api/
+
+## Troubleshooting
+
+Running darktable with Lua debugging enabled provides more information about what is occurring within the scripts.
+
+### Snap
+
+Open a terminal and start darktable with the command `snap run darktable -d lua`. This provides debugging information to give you insight into what is happening.
+
+### Linux and MacOS
+
+Open a terminal and start darktable with the command `darktable -d lua`. This provides debugging information to give you insight into what is happening.
+
+### Windows
+
+Open the Git Bash Shell. Start darktable with the command `/c/Program\ Files/darktable/bin/darktable -d lua`. This provides debugging information to give you insight into what is happening.
+
+## Contributing
+
+In order to have your own scripts added here they have to be under a free license (GPL2+ will definitely work, others can be discussed).

contrib/AutoGrouper.lua

@@ -0,0 +1,158 @@
+--[[AutoGrouper plugin for darktable
+
+  copyright (c) 2019  Kevin Ertel
+  
+  darktable is free software: you can redistribute it and/or modify
+  it under the terms of the GNU General Public License as published by
+  the Free Software Foundation, either version 3 of the License, or
+  (at your option) any later version.
+  
+  darktable is distributed in the hope that it will be useful,
+  but WITHOUT ANY WARRANTY; without even the implied warranty of
+  MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+  GNU General Public License for more details.
+  
+  You should have received a copy of the GNU General Public License
+  along with darktable.  If not, see <http://www.gnu.org/licenses/>.
+]]
+
+--[[About this Plugin
+This plugin adds the module "Auto Group" to darktable's lighttable view
+
+----REQUIRED SOFTWARE----
+None
+
+----USAGE----
+Install: (see here for more detail: https://github.com/darktable-org/lua-scripts )
+ 1) Copy this file in to your "lua/contrib" folder where all other scripts reside. 
+ 2) Require this file in your luarc file, as with any other dt plug-in
+
+Set a gap amount in second which will be used to determine when images should no 
+longer be added to a group. If an image is more then the specified amount of time
+from the last image in the group it will not be added. Images without timestamps 
+in exif data will be ignored.
+
+There are two buttons. One allows the grouping to be performed only on the currently
+selected images, the other button performs grouping on the entire active collection
+]]
+
+local dt = require "darktable"
+local MOD = 'autogrouper'
+local gettext = dt.gettext
+-- Tell gettext where to find the .mo file translating messages for a particular domain
+gettext.bindtextdomain("AutoGrouper",dt.configuration.config_dir.."/lua/locale/")
+local function _(msgid)
+    return gettext.dgettext("AutoGrouper", msgid)
+end
+
+local function InRange(test, low, high) --tests if test value is within range of low and high (inclusive)
+    if test >= low and test <= high then
+        return true
+    else
+        return false
+    end
+end
+
+local function CompTime(first, second) --compares the timestamps and returns true if first was taken before second
+    first_time = first.exif_datetime_taken
+    if string.match(first_time, '[0-9]') == nil then first_time = '9999:99:99 99:99:99' end
+    first_time = tonumber(string.gsub(first_time, '[^0-9]*',''))
+    second_time = second.exif_datetime_taken
+    if string.match(second_time, '[0-9]') == nil then second_time = '9999:99:99 99:99:99' end
+    second_time = tonumber(string.gsub(second_time, '[^0-9]*',''))
+    return first_time < second_time
+end
+
+local function SeperateTime(str) --seperates the timestamp into individual components for used with OS.time operations
+    local cleaned = string.gsub(str, '[^%d]',':')
+    cleaned = string.gsub(cleaned, '::*',':')  --YYYY:MM:DD:hh:mm:ss
+    local year = string.sub(cleaned,1,4)
+    local month = string.sub(cleaned,6,7)
+    local day = string.sub(cleaned,9,10)
+    local hour = string.sub(cleaned,12,13)
+    local min = string.sub(cleaned,15,16)
+    local sec = string.sub(cleaned,18,19)
+    return {year = year, month = month, day = day, hour = hour, min = min, sec = sec}
+end
+
+local function GetTimeDiff(curr_image, prev_image) --returns the time difference (in sec.) from current image and the previous image
+    local curr_time = SeperateTime(curr_image.exif_datetime_taken)
+    local prev_time = SeperateTime(prev_image.exif_datetime_taken)
+    return os.time(curr_time)-os.time(prev_time)
+end
+
+local function main(on_collection)
+    local images = {}
+    if on_collection then 
+        local col_images = dt.collection
+        for i,image in ipairs(col_images) do --copy images to a standard table, table.sort barfs on type dt_lua_singleton_image_collection
+            table.insert(images,i,image)
+        end
+    else
+        images = dt.gui.selection()
+    end
+    dt.preferences.write(MOD, 'active_gap', 'integer', GUI.gap.value)
+    if #images < 2 then 
+        dt.print('please select at least 2 images')
+        return
+    end
+    table.sort(images, function(first, second) return CompTime(first,second) end)  --sort images by timestamp
+    
+    for i, image in ipairs(images) do
+        if i == 1 then 
+            prev_image = image
+        elseif string.match(image.exif_datetime_taken, '[%d]') ~= nil then --make sure current image has a timestamp, if so check if it is within the user specified gap value and add to group
+            local curr_image = image
+            if GetTimeDiff(curr_image, prev_image) <= GUI.gap.value then
+                images[i]:group_with(images[i-1])
+            end
+            prev_image = curr_image
+        end
+    end
+end
+
+-- GUI --
+GUI = {
+    gap =           {},
+    selected =      {},
+    collection =    {}
+}
+temp = dt.preferences.read(MOD, 'active_gap', 'integer')
+if not InRange(temp, 1, 86400) then temp = 3 end
+GUI.gap = dt.new_widget('slider'){
+    label = _('group gap [sec.]'),
+    tooltip = _('minimum gap, in seconds, between groups'),
+    soft_min = 1,
+    soft_max = 60,
+    hard_min = 1,
+    hard_max = 86400,
+    step = 1,
+    digits = 0,
+    value = temp,
+    reset_callback = function(self) 
+        self.value = 3
+    end
+}
+GUI.selected = dt.new_widget("button"){
+    label = _('auto group: selected'),
+    tooltip =_('auto group selected images'),
+    clicked_callback = function() main(false) end
+}
+GUI.collection = dt.new_widget("button"){
+    label = _('auto group: collection'),
+    tooltip =_('auto group the entire collection'),
+    clicked_callback = function() main(true) end
+}
+dt.register_lib(
+    'AutoGroup_Lib',    -- Module name
+    _('auto group'),    -- name
+    true,   -- expandable
+    true,   -- resetable
+    {[dt.gui.views.lighttable] = {"DT_UI_CONTAINER_PANEL_RIGHT_CENTER", 99}},   -- containers
+    dt.new_widget("box"){
+        orientation = "vertical",
+        GUI.gap,
+        GUI.selected,
+        GUI.collection
+        }
+)

contrib/CollectHelper.lua

@@ -0,0 +1,221 @@
+--[[Collect Helper plugin for darktable
+
+  copyright (c) 2019  Kevin Ertel
+  
+  darktable is free software: you can redistribute it and/or modify
+  it under the terms of the GNU General Public License as published by
+  the Free Software Foundation, either version 3 of the License, or
+  (at your option) any later version.
+  
+  darktable is distributed in the hope that it will be useful,
+  but WITHOUT ANY WARRANTY; without even the implied warranty of
+  MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+  GNU General Public License for more details.
+  
+  You should have received a copy of the GNU General Public License
+  along with darktable.  If not, see <http://www.gnu.org/licenses/>.
+]]
+
+--[[About this plugin
+This plugin adds the button(s) to the "Selected Images" module:
+1) Return to Previous Collection
+2) Collect on image's Folder
+3) Collect on image's Color Label(s)
+4) Collect on All (AND)
+
+It also adds 3 preferences to the lua options dialog box which allow the user to activate/deactivate the 3 "Collect on" buttons.
+
+Button Behavior:
+1) Return to Previous Collection - Will reset the collect parameters to the previously active settings
+2) Collect on image's Folder - Will change the collect parameters to be "Folder" with a value of the selected image's folder location
+3) Collect on image's Color Label(s) - Will change the collect parameter to be "Color" with a value of the selected images color labels, will apply multiple parameters with AND logic if multiple exist
+4) Collect on All (AND) - Will collect on all parameters activated by the preferences dialog, as such this button is redundant if you only have one of the two other options enabled
+
+----REQUIRED SOFTWARE----
+NA
+
+----USAGE----
+Install: (see here for more detail: https://github.com/darktable-org/lua-scripts )
+ 1) Copy this file in to your "lua/contrib" folder where all other scripts reside. 
+ 2) Require this file in your luarc file, as with any other dt plug-in
+
+Select the photo you wish to change you collection based on.
+In the "Selected Images" module click on "Collect on this Image"
+
+----KNOWN ISSUES----
+]]
+
+local dt = require "darktable"
+local gettext = dt.gettext
+local previous = nil
+local all_active = false
+
+-- Tell gettext where to find the .mo file translating messages for a particular domain
+gettext.bindtextdomain("CollectHelper",dt.configuration.config_dir.."/lua/locale/")
+
+local function _(msgid)
+    return gettext.dgettext("CollectHelper", msgid)
+end
+
+-- FUNCTION --
+local function CheckSingleImage(selection)
+	if #selection ~= 1 then
+		dt.print(_("Please select a single image"))
+		return true
+	end
+	return false
+end
+local function CheckHasColorLabel(selection)
+	local ret = false
+	for _,image in pairs(selection) do
+		if image.red then ret = true end
+		if image.blue then ret = true end
+		if image.green then ret = true end
+		if image.yellow then ret = true end
+		if image.purple then ret = true end
+	end
+	return ret
+end
+local function PreviousCollection()
+	if previous ~= nil then
+		previous = dt.gui.libs.collect.filter(previous)
+	end
+end
+local function CollectOnFolder(all_rules, all_active)
+	local images = dt.gui.selection()
+	if CheckSingleImage(images) then
+		return
+	end
+	local rules = {}
+	local rule = dt.gui.libs.collect.new_rule()
+	rule.mode = "DT_LIB_COLLECT_MODE_AND"
+	rule.data = images[1].path
+	rule.item = "DT_COLLECTION_PROP_FOLDERS"
+	table.insert(rules, rule)
+	if all_active then
+		for _,active_rule in pairs(rules) do
+			table.insert(all_rules, active_rule)
+		end
+		return all_rules
+	else
+		previous = dt.gui.libs.collect.filter(rules)
+	end
+end
+local function CollectOnColors(all_rules, all_active)
+	local images = dt.gui.selection()
+	if CheckSingleImage(images) then
+		return
+	end
+	if not CheckHasColorLabel(images) then
+		dt.print(_('select an image with an active color label'))
+		return
+	end
+	for _,image in pairs(images) do
+		local rules = {}
+		if image.red then
+			local red_rule = dt.gui.libs.collect.new_rule()
+			red_rule.mode = "DT_LIB_COLLECT_MODE_AND"
+			red_rule.data = "red"
+			red_rule.item = "DT_COLLECTION_PROP_COLORLABEL"
+			table.insert(rules, red_rule)
+		end
+		if image.blue then
+			local blue_rule = dt.gui.libs.collect.new_rule()
+			blue_rule.mode = "DT_LIB_COLLECT_MODE_AND"
+			blue_rule.data = "blue"
+			blue_rule.item = "DT_COLLECTION_PROP_COLORLABEL"
+			table.insert(rules, blue_rule)
+		end
+		if image.green then
+			local green_rule = dt.gui.libs.collect.new_rule()
+			green_rule.mode = "DT_LIB_COLLECT_MODE_AND"
+			green_rule.data = "green"
+			green_rule.item = "DT_COLLECTION_PROP_COLORLABEL"
+			table.insert(rules, green_rule)
+		end
+		if image.yellow then
+			local yellow_rule = dt.gui.libs.collect.new_rule()
+			yellow_rule.mode = "DT_LIB_COLLECT_MODE_AND"
+			yellow_rule.data = "yellow"
+			yellow_rule.item = "DT_COLLECTION_PROP_COLORLABEL"
+			table.insert(rules, yellow_rule)
+		end
+		if image.purple then
+			local purple_rule = dt.gui.libs.collect.new_rule()
+			purple_rule.mode = "DT_LIB_COLLECT_MODE_AND"
+			purple_rule.data = "purple"
+			purple_rule.item = "DT_COLLECTION_PROP_COLORLABEL"
+			table.insert(rules, purple_rule)
+		end
+		if all_active then
+			for _,active_rule in pairs(rules) do
+				table.insert(all_rules, active_rule)
+			end
+			return all_rules
+		else
+			previous = dt.gui.libs.collect.filter(rules)
+		end
+	end
+end
+local function CollectOnAll_AND()
+	local images = dt.gui.selection()
+	if CheckSingleImage(images) then
+		return
+	end
+	local rules = {}
+	if dt.preferences.read('module_CollectHelper','folder','bool') then
+		rules = CollectOnFolder(rules, true)
+	end
+	if dt.preferences.read('module_CollectHelper','colors','bool') then
+		rules = CollectOnColors(rules, true)
+	end
+	previous = dt.gui.libs.collect.filter(rules)
+end
+
+-- GUI --
+dt.gui.libs.image.register_action(
+	_("collect: previous"),
+	function() PreviousCollection() end,
+	_("Sets the Collect parameters to be the previously active parameters")
+)
+if dt.preferences.read('module_CollectHelper','folder','bool') then
+	dt.gui.libs.image.register_action(
+		_("collect: folder"),
+		function() CollectOnFolder(_ , false) end,
+		_("Sets the Collect parameters to be the selected images's folder")
+	)
+end
+if dt.preferences.read('module_CollectHelper','colors','bool') then
+	dt.gui.libs.image.register_action(
+		_("collect: color label(s)"),
+		function() CollectOnColors(_ , false) end,
+		_("Sets the Collect parameters to be the selected images's color label(s)")
+	)
+end
+if dt.preferences.read('module_CollectHelper','all_and','bool') then
+	dt.gui.libs.image.register_action(
+		_("collect: all (AND)"),
+		function() CollectOnAll_AND() end,
+		_("Sets the Collect parameters based on all activated CollectHelper options")
+	)
+end
+
+-- PREFERENCES --
+dt.preferences.register("module_CollectHelper", "all_and",	-- name
+	"bool",	-- type
+	_('CollectHelper: All'),	-- label
+	_('Will create a collect parameter set that utelizes all enabled CollectHelper types (AND)'),	-- tooltip
+	true	-- default
+)
+dt.preferences.register("module_CollectHelper", "colors",	-- name
+	"bool",	-- type
+	_('CollectHelper: Color Label(s)'),	-- label
+	_('Enable the button that allows you to swap to a collection based on selected image\'s COLOR LABEL(S)'),	-- tooltip
+	true	-- default
+)
+dt.preferences.register("module_CollectHelper", "folder",	-- name
+	"bool",	-- type
+	_('CollectHelper: Folder'),	-- label
+	_('Enable the button that allows you to swap to a collection based on selected image\'s FOLDER location'),	-- tooltip
+	true	-- default
+)

contrib/LabelsToTags.lua

@@ -0,0 +1,234 @@
+--[[
+   LABELS TO TAGS
+   Allows the mass-application of tags using color labels and ratings
+   as a guide.
+
+   AUTHOR
+   August Schwerdfeger (august@schwerdfeger.name)
+
+   INSTALLATION
+   * Copy this file into $CONFIGDIR/lua/, where CONFIGDIR
+   is your darktable configuration directory
+   * Add the following line in the file $CONFIGDIR/luarc:
+   require "LabelsToTags"
+
+   ADDITIONAL SOFTWARE NEEDED FOR THIS SCRIPT
+   None.
+
+   USAGE
+   In your 'luarc' file or elsewhere, use the function
+   'register_tag_mapping', defined in this module, to specify
+   one or more tag mappings for use by the module.
+   Any mappings so registered will be selectable, according
+   to their given names, in the module's "mapping" combo box.
+
+   A mapping takes the form of a table mapping patterns to
+   lists of tags. A pattern consists of 6 characters, of which
+   the first five represent color labels and the last the rating.
+   Each color label character may be '+', '-', or '*',
+   indicating that for this pattern to match, the corresponding
+   color label, respectively, must be on, must be off, or can be
+   either. Similarly, the rating character may be a numeral
+   between 0 and 5, "R" for rejected, or "*" for "any value."
+
+   An example call to 'register_tag_mapping' is provided in a
+   comment at the end of this file.
+
+   When the "Start" button is pressed, the module will
+   iterate over each selected image and check the state of
+   that image's color labels and rating against each pattern
+   defined in the selected mapping. For each pattern that
+   matches, the corresponding tags will be added to the
+   image. Any such tag not already existing in the database
+   will be created.
+
+   LICENSE
+   LGPLv2+
+
+]]
+
+local darktable = require("darktable")
+local du = require "lib/dtutils"
+
+du.check_min_api_version("3.0.0", "LabelsToTags") 
+
+-- Lua 5.3 no longer has "unpack" but "table.unpack"
+unpack = unpack or table.unpack
+
+local LIB_ID = "LabelsToTags"
+
+-- Helper functions: BEGIN
+
+local function keySet(t)
+   local rv = {}
+   for k,_ in pairs(t) do
+      table.insert(rv,k)
+   end
+   table.sort(rv)
+   return(rv)
+end
+
+local function generateLabelHash(img)
+   local hash = ""
+   hash = hash .. (img.red and "+" or "-")
+   hash = hash .. (img.yellow and "+" or "-")
+   hash = hash .. (img.green and "+" or "-")
+   hash = hash .. (img.blue and "+" or "-")
+   hash = hash .. (img.purple and "+" or "-")
+   hash = hash .. (img.rating == -1 and "R" or tostring(img.rating))
+   return(hash)
+end
+
+local function hashMatch(hash,pattern)
+   if #(hash) ~= #(pattern) then return(false) end
+   for i = 0,#hash do
+      if string.sub(hash,i,i) ~= string.sub(pattern,i,i) and
+      string.sub(pattern,i,i) ~= "*" then
+	 return(false)
+      end
+   end
+   return(true)
+end
+
+-- Helper functions: END
+
+
+
+
+local initialAvailableMappings = {
+   ["Colors"] = { ["+*****"] = { "Red" },
+      ["*+****"] = { "Yellow" },
+      ["**+***"] = { "Green" },
+      ["***+**"] = { "Blue" },
+      ["****+*"] = { "Purple" } },
+   ["Single colors"] = { ["+----*"] = { "Red", "Only red" },
+      ["-+---*"] = { "Yellow", "Only yellow" },
+      ["--+--*"] = { "Green", "Only green" },
+      ["---+-*"] = { "Blue", "Only blue" },
+      ["----+*"] = { "Purple", "Only purple" } },
+   ["Ratings"] = { ["*****0"] = { "No stars", "Not rejected" },
+      ["*****1"] = { "One star", "Not rejected" },
+      ["*****2"] = { "Two stars", "Not rejected" },
+      ["*****3"] = { "Three stars", "Not rejected" },
+      ["*****4"] = { "Four stars", "Not rejected" },
+      ["*****5"] = { "Five stars", "Not rejected" },
+      ["*****R"] = { "Rejected" } }
+}
+
+local availableMappings = {}
+
+local function getAvailableMappings()
+   if availableMappings == nil or next(availableMappings) == nil then
+      return(initialAvailableMappings)
+   else
+      return(availableMappings)
+   end
+end
+
+local function getComboboxTooltip()
+   if availableMappings == nil or next(availableMappings) == nil then
+      return("No registered mappings -- using defaults")
+   else
+      return("Select a label-to-tag mapping")
+   end
+end
+
+local mappingComboBox = darktable.new_widget("combobox"){
+   label = "mapping",
+   value = 1,
+   tooltip = getComboboxTooltip(),
+   reset_callback = function(selfC)
+      if selfC == nil then
+	 return
+      end
+      i = 1
+      for _,m in pairs(keySet(getAvailableMappings())) do
+	 selfC[i] = m
+	 i = i+1
+      end
+      n = #selfC
+      for j = i,n do
+	 selfC[i] = nil
+      end
+      selfC.value = 1
+      selfC.tooltip = getComboboxTooltip()
+   end,
+   unpack(keySet(getAvailableMappings()))
+}
+
+local function doTagging(selfC)
+   local job = darktable.gui.create_job(string.format("labels to tags (%d image" .. (#(darktable.gui.action_images) == 1 and "" or "s") .. ")",#(darktable.gui.action_images)),true)
+   job.percent = 0.0
+   local pctIncrement = 1.0 / #(darktable.gui.action_images)
+
+   local availableMappings = getAvailableMappings()
+   local memoizedTags = {}
+   for _,img in ipairs(darktable.gui.action_images) do
+      local tagsToApply = {}
+      local hash = generateLabelHash(img)
+      for k,v in pairs(availableMappings[mappingComboBox.value]) do
+	 if hashMatch(hash,k) then
+	    for _,tag in ipairs(v) do
+	       tagsToApply[tag] = true
+	    end
+	 end
+      end
+      for k,_ in pairs(tagsToApply) do
+	 if memoizedTags[k] == nil then
+	    memoizedTags[k] = darktable.tags.create(k)
+	 end
+	 darktable.tags.attach(memoizedTags[k],img)
+      end
+      job.percent = job.percent + pctIncrement
+   end
+   job.valid = false
+end
+
+local my_widget = darktable.new_widget("box") {
+   orientation = "vertical",
+   mappingComboBox,
+   darktable.new_widget("button") {
+      label = "start",
+      tooltip = "Tag all selected images",
+      clicked_callback = doTagging
+   }
+}
+
+local PATTERN_PATTERN = "^[+*-][+*-][+*-][+*-][+*-][0-5R*]$"
+
+darktable.register_tag_mapping = function(name, mapping)
+   if availableMappings[name] ~= nil then
+      darktable.print_error("Tag mapping '" .. name .. "' already registered")
+      return
+   end
+   for pattern,tags in pairs(mapping) do
+      if string.match(pattern,PATTERN_PATTERN) == nil then
+	 darktable.print_error("In tag mapping '" .. name .. "': Pattern '" .. pattern .. "' invalid")
+	 return
+      end
+      for _,tag in ipairs(tags) do
+	 if type(tag) ~= "string" then
+	    darktable.print_error("In tag mapping '" .. name .. "': All tag mappings must be lists of strings")
+	    return
+	 end
+      end
+   end
+   availableMappings[name] = mapping
+   mappingComboBox.reset_callback(mappingComboBox)
+end
+
+--[[
+darktable.register_tag_mapping("Example",
+			       { ["+----*"] = { "Red", "Only red" },
+				 ["-+---*"] = { "Yellow", "Only yellow" },
+				 ["--+--*"] = { "Green", "Only green" },
+				 ["---+-*"] = { "Blue", "Only blue" },
+				 ["****+*"] = { "Purple" },
+				 ["----+*"] = { "Only purple" },
+				 ["*****1"] = { "One star" },
+				 ["*****R"] = { "Rejected" } })
+]]
+
+darktable.register_lib(LIB_ID,"labels to tags",true,true,{
+			  [darktable.gui.views.lighttable] = {"DT_UI_CONTAINER_PANEL_RIGHT_CENTER",20},
+								 },my_widget,nil,nil)

contrib/OpenInExplorer.lua

@@ -0,0 +1,113 @@
+--[[
+OpenInExplorer plugin for darktable
+
+  copyright (c) 2018  Kevin Ertel
+  
+  darktable is free software: you can redistribute it and/or modify
+  it under the terms of the GNU General Public License as published by
+  the Free Software Foundation, either version 3 of the License, or
+  (at your option) any later version.
+  
+  darktable is distributed in the hope that it will be useful,
+  but WITHOUT ANY WARRANTY; without even the implied warranty of
+  MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+  GNU General Public License for more details.
+  
+  You should have received a copy of the GNU General Public License
+  along with darktable.  If not, see <http://www.gnu.org/licenses/>.
+]]
+
+--[[About this plugin
+This plugin adds the module "OpenInExplorer" to darktable's lighttable view
+
+----REQUIRED SOFTWARE----
+Microsoft Windows or Linux with installed Nautilus
+
+----USAGE----
+Install: (see here for more detail: https://github.com/darktable-org/lua-scripts )
+ 1) Copy this file in to your "lua/contrib" folder where all other scripts reside. 
+ 2) Require this file in your luarc file, as with any other dt plug-in
+
+Select the photo(s) you wish to find in explorer and press "Go to Folder". 
+A file explorer window will be opened for each selected file at the file's location; the file will be highlighted.
+
+----KNOWN ISSUES----
+]]
+
+local dt = require "darktable"
+local du = require "lib/dtutils"
+local df = require "lib/dtutils.file"
+local dsys = require "lib/dtutils.system"
+
+--Check API version
+du.check_min_api_version("5.0.0", "OpenInExplorer") 
+
+local PS = dt.configuration.running_os == "windows" and  "\\"  or  "/"
+
+--Detect OS and modify accordingly--	
+local proper_install = false
+if dt.configuration.running_os ~= "macos" then
+  proper_install = true
+else
+  dt.print_error('OpenInExplorer plug-in only supports Windows and Linux at this time')
+  return
+end
+
+
+-- FUNCTIONS --
+
+local function open_in_explorer() --Open in Explorer
+  local images = dt.gui.selection()
+  local curr_image = ""
+  if #images == 0 then
+    dt.print('please select an image')
+  elseif #images <= 15 then
+    for _,image in pairs(images) do 
+      curr_image = image.path..PS..image.filename
+      local run_cmd = "explorer.exe /select, "..curr_image
+      dt.print_log("OpenInExplorer run_cmd = "..run_cmd)
+      resp = dsys.external_command(run_cmd)
+    end
+  else
+    dt.print('please select fewer images (max 15)')
+  end
+end
+
+local function open_in_nautilus() --Open in Nautilus
+  local images = dt.gui.selection()
+  local curr_image = ""
+  if #images == 0 then
+    dt.print('please select an image')
+  elseif #images <= 15 then
+    for _,image in pairs(images) do 
+      curr_image = image.path..PS..image.filename
+      local run_cmd = [[busctl --user call org.freedesktop.FileManager1 /org/freedesktop/FileManager1 org.freedesktop.FileManager1 ShowItems ass 1 ]] .. df.sanitize_filename("file://"..curr_image) .. [[ ""]]
+      dt.print_log("OpenInExplorer run_cmd = "..run_cmd)
+      resp = dsys.external_command(run_cmd)
+    end
+  else
+    dt.print('please select fewer images (max 15)')
+  end
+end
+
+local function open_in_filemanager() --Open
+  --Inits--
+  if not proper_install then
+    return
+  end
+
+  if (dt.configuration.running_os == "windows") then
+    open_in_explorer()
+  elseif (dt.configuration.running_os == "linux") then
+    open_in_nautilus()
+  end  
+end
+
+-- GUI --
+if proper_install then
+  dt.gui.libs.image.register_action(
+    "show in file explorer",
+    function() open_in_filemanager() end,
+    "Opens File Explorer at the selected image's location"
+  )
+end

contrib/autostyle.lua

@@ -28,6 +28,8 @@ USAGE
   > exiftool -AutoDynamicRange XE021351.RAF
   Auto Dynamic Range              : 200%
 
+ADDITIONAL SOFTWARE NEEDED FOR THIS SCRIPT
+* exiftool
 
 LICENSE
 GPLv2
@@ -35,13 +37,14 @@ GPLv2
 ]]
 
 local darktable = require "darktable"
-
+local du = require "lib/dtutils"
+local filelib = require "lib/dtutils.file"
 
 -- Forward declare the functions
 local autostyle_apply_one_image,autostyle_apply_one_image_event,autostyle_apply,exiftool_attribute,capture
 
 -- Tested it with darktable 1.6.1 and darktable git from 2014-01-25
-darktable.configuration.check_version(...,{2,0,2},{2,1,0},{3,0,0})
+du.check_min_api_version("2.0.2", "autostyle") 
 
 -- Receive the event triggered
 function autostyle_apply_one_image_event(event,image)
@@ -56,16 +59,21 @@ function autostyle_apply_one_image (image)
   -- check they all exist (correct syntax)
   if (not tag) then
 	  darktable.print("EXIF TAG not found in " .. darktable.preferences.read("autostyle","exif_tag","string"))
-	  return
+	  return 0
   end
   if (not value) then
 	  darktable.print("value to match not found in " .. darktable.preferences.read("autostyle","exif_tag","string"))
-	  return
+	  return 0
   end
   if (not style_name) then
 	  darktable.print("style name not found in " .. darktable.preferences.read("autostyle","exif_tag","string"))
-	  return
+	  return 0
   end
+  if not filelib.check_if_bin_exists("exiftool") then
+	  darktable.print("Can't find exiftool")
+          return 0
+  end
+	
 	
   -- First find the style (we have its name)
   local styles= darktable.styles
@@ -77,40 +85,50 @@ function autostyle_apply_one_image (image)
   end
   if (not style) then
 	  darktable.print("style not found for autostyle: " .. style_name)
+          return 0
   end
 
   -- Apply the style to image, if it is tagged
   local ok,auto_dr_attr= pcall(exiftool_attribute,image.path .. '/' .. image.filename,tag)
+  --darktable.print_error("dr_attr:" .. auto_dr_attr)
   -- If the lookup fails, stop here
   if (not ok) then
-    return
+    darktable.print("Couldn't get attribute " .. auto_dr_attr .. " from exiftool's output")
+    return 0
   end
   if auto_dr_attr==value then
---	  darktable.print("Image " .. image.filename .. ": autostyle automatically applied " .. darktable.preferences.read("autostyle","exif_tag","string") )
+	  darktable.print_log("Image " .. image.filename .. ": autostyle automatically applied " .. darktable.preferences.read("autostyle","exif_tag","string") )
 	  darktable.styles.apply(style,image)
---  else
---	  darktable.print("Image " .. image.filename .. ": autostyle not applied, exif tag " .. darktable.preferences.read("autostyle","exif_tag","string")  .. " not matched: " .. auto_dr_attr)
+	  return 1
+  else
+	  darktable.print_log("Image " .. image.filename .. ": autostyle not applied, exif tag " .. darktable.preferences.read("autostyle","exif_tag","string")  .. " not matched: " .. auto_dr_attr)
+	  return 0
   end
 end 
 
 
 function autostyle_apply( shortcut)
   local images = darktable.gui.action_images
+  local images_processed =0
+  local images_submitted =0
   for _,image in pairs(images) do
-    autostyle_apply_one_image(image)
+    images_submitted = images_submitted +1
+    images_processed = images_processed + autostyle_apply_one_image(image)
   end
+  darktable.print("Applied auto style to " .. images_processed .. " out of " .. images_submitted .. " image(s)")
 end
 
 -- Retrieve the attribute through exiftool
 function exiftool_attribute(path,attr)
-  local cmd="exiftool -" .. attr .. " '" ..path.. "'";
+  local cmd="exiftool -" .. attr .. " '" .. path .. "'";
   local exifresult=get_stdout(cmd)
   local attribute=string.match(exifresult,": (.*)")
   if (attribute == nil) then
-    darktable.print( "Could not find the attribute " .. attr .. " using the command: <" .. cmd .. ">")
+    darktable.print_error( "Could not find the attribute " .. attr .. " using the command: <" .. cmd .. ">")
     -- Raise an error to the caller
     error( "Could not find the attribute " .. attr .. " using the command: <" .. cmd .. ">");
   end
+--  darktable.print_error("Returning attribute: " .. attribute)
   return attribute
 end
 
@@ -118,7 +136,7 @@ end
 function get_stdout(cmd)
   -- Open the command, for reading
   local fd = assert(io.popen(cmd, 'r'))
-  dt.control.read(fd)
+  darktable.control.read(fd)
   -- slurp the whole file
   local data = assert(fd:read('*a'))
 

contrib/calcDistance.lua

@@ -1,106 +0,0 @@
---[[
-    This file is part of darktable,
-    Copyright 2014-2016 by Tobias Jakobs.
-
-    This program is free software: you can redistribute it and/or modify
-    it under the terms of the GNU General Public License as published by
-    the Free Software Foundation; either version 3 of the License, or
-    (at your option) any later version.
-
-    This program is distributed in the hope that it will be useful,
-    but WITHOUT ANY WARRANTY; without even the implied warranty of
-    MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
-    GNU General Public License for more details.
-
-    You should have received a copy of the GNU General Public License
-    along with this program.  If not, see <http://www.gnu.org/licenses/>.
-]]
---[[
-darktable calc distance script
-
-This script calcs the distance between two images using the GPS data in the metadata
-
-USAGE
-* require this script from your main lua file
-* register a shortcut
-]]
-   
-local dt = require "darktable"
-local gettext = dt.gettext
-dt.configuration.check_version(...,{3,0,0})
-
--- Tell gettext where to find the .mo file translating messages for a particular domain
-gettext.bindtextdomain("calcDistance",dt.configuration.config_dir.."/lua/")
-
-local function _(msgid)
-    return gettext.dgettext("calcDistance", msgid)
-end
-
-local function calcDistance()
-	local sel_images = dt.gui.selection()
-
-    local lat1 = 0;
-    local lon1 = 0;
-    local lat2 = 0;
-    local lon2 = 0;
-    local ele1 = 0;
-    local ele2 = 0;
-
-    local i = 0;
-
-    local sel_images = dt.gui.selection()
-
-    for _,image in ipairs(sel_images) do
-	if ((image.longitude and image.latitude) and 
-            (image.longitude ~= 0 and image.latitude ~= 90) -- Sometimes the north-pole but most likely just wrong data
-           ) then
-          
-          i = i + 1;
-          
-          if (i == 1) then
-            lat1 = image.latitude;
-            lon1 = image.longitude;
-            ele1 = image.elevation;
-          end
-         
-         lat2 = image.latitude;
-         lon2 = image.longitude;
-         ele2 = image.elevation;
-
-        end
-    end
-
--- I used code from here:
--- http://stackoverflow.com/questions/27928/how-do-i-calculate-distance-between-two-latitude-longitude-points
-
-    local earthRadius = 6371; -- Radius of the earth in km
-    local dLat = math.rad(lat2-lat1);  -- deg2rad below
-    local dLon = math.rad(lon2-lon1); 
-    local a = 
-      math.sin(dLat/2) * math.sin(dLat/2) +
-      math.cos(math.rad(lat1)) * math.cos(math.rad(lat2)) * 
-      math.sin(dLon/2) * math.sin(dLon/2)
-      ; 
-    local angle = 2 * math.atan2(math.sqrt(a), math.sqrt(1-a)); 
-    local distance = earthRadius * angle; -- Distance in km  
-
-    -- Add the elevation to the calculation
-    local elevation = 0;
-    elevation = math.abs(ele1 - ele2) / 1000;  --in km
-    distance = math.sqrt(math.pow(elevation,2) + math.pow(distance,2) );
-
-    local distanceUnit
-    if (distance < 1) then
-        distance = distance * 1000
-        distanceUnit = "m"
-    else
-        distanceUnit = "km"
-    end
-    dt.print(string.format("Distance: %.2f %s", distance, distanceUnit))
-
-
-
-end
-
--- Register
-dt.register_event("shortcut",calcDistance,_("Calculate the distance from latitude and longitude in km"))

contrib/clear_GPS.lua

@@ -37,16 +37,17 @@
 ]]
 
 local dt = require "darktable"
+local du = require "lib/dtutils"
 local gettext = dt.gettext
 
 -- not a number
 local NaN = 0/0
 
-dt.configuration.check_version(...,{3,0,0})
+du.check_min_api_version("3.0.0", "clear_GPS") 
 
 
 -- Tell gettext where to find the .mo file translating messages for a particular domain
-gettext.bindtextdomain("clear_GPS",dt.configuration.config_dir.."/lua/")
+gettext.bindtextdomain("clear_GPS",dt.configuration.config_dir.."/lua/locale/")
 
 local function _(msgid)
     return gettext.dgettext("clear_GPS", msgid)

contrib/copy_attach_detach_tags.lua

@@ -33,15 +33,19 @@ USAGE
    darktable internal tags starting with 'darktable|' will not be touched
  * <your shortcut3> removes all expect darktable internal tags from selected images
  * <your shortcut4> replaces all tags expect darktable internals
-
+ * A module reset will empty the clipboard
 ]]
 
 local dt = require "darktable"
+local du = require "lib/dtutils"
+local debug = require "darktable.debug"
+
 local gettext = dt.gettext
-dt.configuration.check_version(...,{3,0,0},{4,0,0})
+
+du.check_min_api_version("3.0.0", "copy_attach_detach_tags") 
 
 -- Tell gettext where to find the .mo file translating messages for a particular domain
-gettext.bindtextdomain("copy_attach_detach_tags",dt.configuration.config_dir.."/lua/")
+gettext.bindtextdomain("copy_attach_detach_tags",dt.configuration.config_dir.."/lua/locale/")
 
 local function _(msgid)
     return gettext.dgettext("copy_attach_detach_tags", msgid)
@@ -156,43 +160,58 @@ end
 taglist_label.reset_callback = mcopy_tags
 
 -- create buttons and elements
-local mc_button = dt.new_widget("button") {
-         label = _('multi copy tags'),
-         clicked_callback = mcopy_tags}
-
-local p_button = dt.new_widget("button") {
-         label = _('paste tags'),
-         clicked_callback = attach_tags}
-
-local rep_button = dt.new_widget("button") {
-         label = _('replace tags'),
-         clicked_callback = replace_tags}
-
-local rem_button = dt.new_widget("button") {
-         label = _('remove all tags'),
-         clicked_callback = detach_tags}
-
 local taglabel = dt.new_widget("label") {
          label = _('tag clipboard'),
          selectable = false,
          ellipsize = "middle",
          halign = "start"}
 
+local box1 = dt.new_widget("box"){
+                  orientation = "horizontal",
+                  dt.new_widget("button") {
+                  label = _('multi copy tags'),
+                  clicked_callback = mcopy_tags},
+                  dt.new_widget("button") {
+                  label = _('paste tags'),
+                  clicked_callback = attach_tags}
+
+                  }
+
+local box2 = dt.new_widget("box"){
+                  orientation = "horizontal",
+                  dt.new_widget("button") {
+                  label = _('replace tags'),
+                  clicked_callback = replace_tags},
+                  dt.new_widget("button") {
+                  label = _('remove all tags'),
+                  clicked_callback = detach_tags}
+                  }
+
+local sep = dt.new_widget("separator"){}
+
+-- pack elements into widget table for a nicer layout
+local widget_table = {}
+
+widget_table[1] = box1
+widget_table[#widget_table+1] = box2
+
+widget_table[#widget_table+1] = sep
+widget_table[#widget_table+1] = taglabel
+widget_table[#widget_table+1] = taglist_label
+
 
 -- create modul
-dt.register_lib("tagging_addon","Tagging addon",true,false,{
+dt.register_lib("tagging_addon","Tagging addon",true,true,{
     [dt.gui.views.lighttable] = {"DT_UI_CONTAINER_PANEL_RIGHT_CENTER",500}
     },
     dt.new_widget("box") {
-      orientation = "vertical",
-      mc_button,
-      p_button,
-      rep_button,
-      rem_button,
-      taglabel,
-      dt.new_widget("separator"){},
-      taglist_label
-    },
+  --    orientation = "vertical",
+      reset_callback = function()
+                    taglist_label.label = ""
+                    image_tags = {}
+                    end,
+      table.unpack(widget_table),
+      },
    nil,
    nil
   )

contrib/cr2hdr.lua

@@ -33,9 +33,10 @@ USAGE
 ]]
 
 local darktable = require "darktable"
+local du = require "lib/dtutils"
 
 -- Tested with darktable 2.0.1
-darktable.configuration.check_version(...,{2,0,0},{3,0,0})
+du.check_min_api_version("2.0.0", "cr2hdr")
 
 local queue = {}
 local processed_files = {}
@@ -60,7 +61,7 @@ end
 local function convert_image(image)
     if string.sub(image.filename, -3) == "CR2" then
         local filename = image.path .. "/" .. image.filename
-        local result = dt.control.execute( "cr2hdr " .. filename)
+        local result = darktable.control.execute( "cr2hdr " .. filename)
         local out_filename = string.gsub(filename, ".CR2", ".DNG")
         local file = io.open(out_filename)
         if file then

contrib/fujifilm_ratings.lua

@@ -0,0 +1,71 @@
+--[[ fujifilm_ratings-0.1
+
+Support for importing Fujifilm in-camera ratings in darktable.
+
+Copyright (C) 2017 Ben Mendis <ben.mendis@gmail.com>
+
+This program is free software; you can redistribute it and/or modify
+it under the terms of the GNU General Public License as published by
+the Free Software Foundation; either version 2 of the License, or
+(at your option) any later version.
+
+This program is distributed in the hope that it will be useful,
+but WITHOUT ANY WARRANTY; without even the implied warranty of
+MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+GNU General Public License for more details.
+
+You should have received a copy of the GNU General Public License along
+with this program; if not, write to the Free Software Foundation, Inc.,
+51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA.
+
+Dependencies:
+- exiftool (https://www.sno.phy.queensu.ca/~phil/exiftool/)
+
+--]]
+
+local dt = require "darktable"
+local du = require "lib/dtutils"
+local df = require "lib/dtutils.file"
+local gettext = dt.gettext
+
+du.check_min_api_version("4.0.0", "fujifilm_ratings")
+
+gettext.bindtextdomain("fujifilm_ratings", dt.configuration.config_dir.."/lua/locale/")
+
+local function _(msgid)
+	return gettext.dgettext("fujifilm_ratings", msgid)
+end
+
+local function detect_rating(event, image)
+	if not df.check_if_bin_exists("exiftool") then
+		dt.print_error(_("exiftool not found"))
+		return
+	end
+	local RAF_filename = tostring(image)
+	local JPEG_filename = string.gsub(RAF_filename, "%.RAF$", ".JPG")
+	local command = "exiftool -Rating " .. JPEG_filename
+	dt.print_error(command)
+	local output = io.popen(command)
+	local jpeg_result = output:read("*all")
+	output:close()
+	if string.len(jpeg_result) > 0 then
+		jpeg_result = string.gsub(jpeg_result, "^Rating.*(%d)", "%1")
+		image.rating = tonumber(jpeg_result)
+		dt.print_error(_("Using JPEG Rating: ") .. tostring(jpeg_result))
+		return
+	end
+	command = "exiftool -Rating " .. RAF_filename
+	dt.print_error(command)
+	output = io.popen(command)
+	local raf_result = output:read("*all")
+	output:close()
+	if string.len(raf_result) > 0 then
+		raf_result = string.gsub(raf_result, "^Rating.*(%d)", "%1")
+		image.rating = tonumber(raf_result)
+		dt.print_error(_("Using RAF Rating: ") .. tostring(raf_result))
+	end
+end
+
+dt.register_event("post-import-image", detect_rating)
+
+print(_("fujifilm_ratings loaded."))

contrib/geoJSON_export.lua

@@ -33,13 +33,15 @@ USAGE
 ]]
 
 local dt = require "darktable"
+local du = require "lib/dtutils"
+local df = require "lib/dtutils.file"
 require "official/yield"
-
 local gettext = dt.gettext
-dt.configuration.check_version(...,{3,0,0})
+
+du.check_min_api_version("3.0.0", "geoJSON_export") 
 
 -- Tell gettext where to find the .mo file translating messages for a particular domain
-gettext.bindtextdomain("geoJSON_export",dt.configuration.config_dir.."/lua/")
+gettext.bindtextdomain("geoJSON_export",dt.configuration.config_dir.."/lua/locale/")
 
 local function _(msgid)
     return gettext.dgettext("geoJSON_export", msgid)
@@ -73,37 +75,27 @@ local function show_status(storage, image, format, filename, number, total, high
     dt.print(string.format(_("Export Image %i/%i"), number, total))
 end
 
-local function checkIfBinExists(bin)
-    local handle = io.popen("which "..bin)
-    local result = handle:read()
-    local ret
-    handle:close()
-    if (not result) then
-        dt.print_error(bin.." not found")
-        ret = false
-    end
-    ret = true
-    return ret
-end
-
 local function create_geoJSON_file(storage, image_table, extra_data)
-
-    if not checkIfBinExists("mkdir") then
+    if not df.check_if_bin_exists("mkdir") then
+        dt.print_error(_("mkdir not found"))
         return
     end
-    if not checkIfBinExists("convert") then
+    if not df.check_if_bin_exists("convert") then
+        dt.print_error(_("convert not found"))
         return
     end
-    if not checkIfBinExists("xdg-open") then
+    if not df.check_if_bin_exists("xdg-open") then
+        dt.print_error(_("xdg-open not found"))
         return
     end
-    if not checkIfBinExists("xdg-user-dir") then
+    if not df.check_if_bin_exists("xdg-user-dir") then
+        dt.print_error(_("xdg-user-dir not found"))
         return
     end
 
     dt.print_error("Will try to export geoJSON file now")
 
-    local xportDirectory = dt.preferences.read("geoJSON_export","ExportDirectory","string")
+    local exportDirectory = dt.preferences.read("geoJSON_export","ExportDirectory","string")
 
     -- Creates dir if not exsists
     local imageFoldername = "files/"
@@ -115,8 +107,8 @@ local function create_geoJSON_file(storage, image_table, extra_data)
         if ((image.longitude and image.latitude) and
             (image.longitude ~= 0 and image.latitude ~= 90) -- Sometimes the north-pole but most likely just wrong data
            ) then
-            local path, filename, filetype = string.match(image, "(.-)([^\\/]-%.?([^%.\\/]*))$")
-	    filename = string.upper(string.gsub(filename,"%.", "_"))
+            local path, filename, filetype = string.match(exported_image, "(.-)([^\\/]-%.?([^%.\\/]*))$")
+            filename = string.upper(string.gsub(filename,"%.%w*", ""))
 
             -- convert -size 92x92 filename.jpg -resize 92x92 +profile "*" thumbnail.jpg
             --	In this example, '-size 120x120' gives a hint to the JPEG decoder that the image is going to be downscaled to
@@ -131,8 +123,8 @@ local function create_geoJSON_file(storage, image_table, extra_data)
             dt.control.execute( concertCommand)
         end
 
-        -- delete the original image to not get into the kmz file
-        os.remove(image)
+        -- delete the original image
+        os.remove(exported_image)
 
         local pattern = "[/]?([^/]+)$"
         filmName = string.match(image.film.path, pattern)
@@ -149,11 +141,24 @@ local function create_geoJSON_file(storage, image_table, extra_data)
 ]]
 
     for image,exported_image in pairs(image_table) do
-	filename = string.upper(string.gsub(image.filename,"%.", "_"))
+	--filename = string.upper(string.gsub(image.filename,"%.", "_"))
+        -- Extract filename, e.g DSC9784.ARW -> DSC9784
+        filename = string.upper(string.gsub(image.filename,"%.%w*", ""))
+        -- Extract extension from exported image (user can choose JPG or PNG), e.g DSC9784.JPG -> .JPG
+        extension = string.match(exported_image,"%.%w*$")
 
 	if ((image.longitude and image.latitude) and
             (image.longitude ~= 0 and image.latitude ~= 90) -- Sometimes the north-pole but most likely just wrong data
            ) then
+
+            local image_title, image_description
+            if (image.title and image.title ~= "") then
+                image_title = image.title
+            else
+                image_title = image.filename
+            end
+            image_description = image.description
+
             geoJSON_file = geoJSON_file..
 [[    {
       "type": "Feature",
@@ -164,8 +169,8 @@ local function create_geoJSON_file(storage, image_table, extra_data)
             geoJSON_file = geoJSON_file..
 [[    },
       "properties": {
-        "title": "]]..image.title..[[",
-        "description": "]]..image.description..[[",
+        "title": "]]..image_title..[[",
+        "description": "]]..image_description..[[",
         "image": "]]..imageFoldername..filename..[[.jpg",
         "icon": {
           "iconUrl": "]]..imageFoldername.."thumb_"..filename..[[.jpg",
@@ -196,7 +201,7 @@ local function create_geoJSON_file(storage, image_table, extra_data)
 <html>
 <head>
   <meta charset=utf-8 />
-  <title>2014-05-31 Rieselfelder</title>
+  <title>]]..filmName..[[</title>
   <script src='https://api.tiles.mapbox.com/mapbox.js/v2.2.4/mapbox.js'></script>
   <link href='https://api.tiles.mapbox.com/mapbox.js/v2.2.4/mapbox.css' rel='stylesheet' />
   <style>
@@ -310,6 +315,9 @@ dt.preferences.register("geoJSON_export",
 local handle = io.popen("xdg-user-dir DESKTOP")
 local result = handle:read()
 handle:close()
+if (result == nil) then
+	result = ""
+end
 dt.preferences.register("geoJSON_export",
 	"ExportDirectory",
 	"directory",

contrib/geoToolbox.lua

@@ -0,0 +1,688 @@
+--[[
+    This file is part of darktable,
+    copyright (c) 2016 Tobias Jakobs
+
+    darktable is free software: you can redistribute it and/or modify
+    it under the terms of the GNU General Public License as published by
+    the Free Software Foundation, either version 3 of the License, or
+    (at your option) any later version.
+
+    darktable is distributed in the hope that it will be useful,
+     but WITHOUT ANY WARRANTY; without even the implied warranty of
+    MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+    GNU General Public License for more details.
+
+    You should have received a copy of the GNU General Public License
+    along with darktable.  If not, see <http://www.gnu.org/licenses/>.
+]]
+--[[
+
+USAGE
+* require this script from your main lua file
+  To do this add this line to the file .config/darktable/luarc:
+require "geoToolbox"
+
+* it creates a new geoToolbox lighttable module
+]]
+
+local dt = require "darktable"
+local du = require "lib/dtutils"
+local df = require "lib/dtutils.file"
+require "official/yield"
+local gettext = dt.gettext
+
+du.check_min_api_version("3.0.0", "geoToolbox") 
+
+-- Tell gettext where to find the .mo file translating messages for a particular domain
+gettext.bindtextdomain("geoToolbox",dt.configuration.config_dir.."/lua/locale/")
+
+local function _(msgid)
+    return gettext.dgettext("geoToolbox", msgid)
+end
+
+-- <GUI>
+local labelDistance = dt.new_widget("label")
+labelDistance.label = _("Distance:")
+
+local label_copy_gps_lat = dt.new_widget("check_button")
+{
+  label = _("latitude:"),
+  value = true
+}
+local label_copy_gps_lon = dt.new_widget("check_button")
+{
+  label = _("longitude:"),
+  value = true
+}
+local label_copy_gps_ele = dt.new_widget("check_button")
+{
+  label = _("elevation:"),
+  value = true
+}
+-- </GUI>
+
+local function select_with_gps()
+  local selection = {}
+  job = dt.gui.create_job(_("GPS selection"), true, stop_selection)
+
+  for key,image in ipairs(dt.collection) do
+    if job.valid then
+      job.percent = (key-1)/#dt.collection
+
+      if (image.longitude and image.latitude) then
+        table.insert(selection,image)
+      end
+    else
+      break
+    end
+  end
+
+  dt.gui.selection(selection)
+  job.valid = false
+end
+
+local function select_without_gps()
+  local selection = {}
+  job = dt.gui.create_job(_("GPS selection"), true, stop_selection)
+
+  for key,image in ipairs(dt.collection) do
+    if job.valid then
+      job.percent = (key-1)/#dt.collection
+
+      if (not image.longitude and not image.latitude) then
+        table.insert(selection,image)
+      end
+    else
+      break
+    end
+  end
+  dt.gui.selection(selection)
+  job.valid = false
+end
+
+local function stop_selection(job)
+    job.valid = false
+end
+
+-- This is used for older images in the DB
+function isnan(x) return x ~= x end
+
+-- Function from:
+-- https://forums.coronalabs.com/topic/29019-convert-string-to-date/
+-- This looks much scarier than it really is. The goal is to turn that string
+-- into a Unix timestamp (number of seconds since Jan 1, 1970, the standard
+-- used by most systems). So we use the string.match() method to fetch the
+-- various date and time parts into their own variables: xyear, xmonth,etc.
+function make_time_stamp(dateString)
+  local convertedTimestamp
+--dt.print_error(dateString)
+  if (dateString) then
+    local pattern = "(%d+)%:(%d+)%:(%d+) (%d+):(%d+):(%d+)"
+    local xyear, xmonth, xday, xhour, xminute, xseconds = dateString:match(pattern)
+--dt.print_error(xyear)
+    convertedTimestamp = os.time({
+        year = xyear,
+        month = xmonth,
+        day = xday,
+        hour = xhour,
+        min = xminute,
+        sec = xseconds})
+  else
+    convertedTimestamp = 0
+  end
+
+  return convertedTimestamp
+end
+
+local first_have_data = false
+local first_latitude = ''
+local first_longitude = ''
+local first_elevation = ''
+local first_image_date = 0
+
+local function get_first_coordinate()
+  local sel_images = dt.gui.action_images
+
+  first_latitude = ''
+  first_longitude = ''
+  first_elevation = ''
+  first_image_date = 0
+
+  for _,image in ipairs(sel_images) do
+    if not image then
+      first_have_data = false
+    else
+      image_date = image.exif_datetime_taken
+      first_have_data = true
+      if (image.latitude) then
+        first_latitude = image.latitude
+      end
+      if (image.longitude) then
+          first_longitude = image.longitude
+      end
+      if (image.elevation) then
+          first_elevation = image.elevation
+      end
+      if (image.exif_datetime_taken) then
+          first_image_date = make_time_stamp(image.exif_datetime_taken)
+--dt.print_error(image.exif_datetime_taken)
+--dt.print_error(first_image_date)
+      end
+    end
+    return
+  end
+end
+
+local second_have_data = false
+local second_latitude = ''
+local second_longitude = ''
+local second_elevation = ''
+local second_image_date = 0
+
+local function get_second_coordinate()
+  local sel_images = dt.gui.action_images
+
+  second_latitude = ''
+  second_longitude = ''
+  second_elevation = ''
+  second_image_date = 0
+
+  for _,image in ipairs(sel_images) do
+    if not image then
+      second_have_data = false
+    else
+      image_date = image.exif_datetime_taken
+      second_have_data = true
+      if (image.latitude) then
+        second_latitude = image.latitude
+      end
+      if (image.longitude) then
+          second_longitude = image.longitude
+      end
+      if (image.elevation) then
+          second_elevation = image.elevation
+      end
+      if (image.exif_datetime_taken) then
+          second_image_date = make_time_stamp(image.exif_datetime_taken)
+      end
+    end
+    return
+  end
+end
+
+local calc_in_between_slider = dt.new_widget("slider")
+{
+  label = "Position between",
+  soft_min = 0,      -- The soft minimum value for the slider, the slider can't go beyond this point
+  soft_max = 100,     -- The soft maximum value for the slider, the slider can't go beyond this point
+  hard_min = -100,       -- The hard minimum value for the slider, the user can't manually enter a value beyond this point
+  hard_max = 200,    -- The hard maximum value for the slider, the user can't manually enter a value beyond this point
+  value = 50          -- The current value of the slider
+}
+
+--ToDo: this needs more love
+local function calc_in_between()
+  local sel_images = dt.gui.action_images
+  for _,image in ipairs(sel_images) do
+    if image then
+      image_date = make_time_stamp(image.exif_datetime_taken)
+      if (first_have_data and second_have_data) then
+        local start_new      = 0
+        local end_new        = second_image_date - first_image_date
+        local image_date_new = image_date - first_image_date
+
+        local percent_in_between
+        if (end_new == 0) then
+          percent_in_between = 1
+        else
+          percent_in_between = image_date_new/end_new
+        end
+        calc_in_between_slider.value = percent_in_between * 100
+dt.print_error(percent_in_between)
+        local in_between_latitude  = first_latitude + (second_latitude - first_latitude) * percent_in_between
+        local in_between_longitude = first_longitude + (second_longitude - first_longitude) * percent_in_between
+
+        if (first_elevation and second_elevation) then
+          local in_between_elevation = first_elevation + (second_elevation - first_elevation) * percent_in_between
+
+dt.print_error(first_image_date)
+dt.print_error(second_image_date)
+dt.print_error(image_date)
+dt.print_error(first_elevation)
+dt.print_error(second_elevation)
+dt.print_error(in_between_elevation)
+
+        end
+      end
+    end
+  end
+end
+
+local copy_gps_have_data = false
+local copy_gps_latitude = ''
+local copy_gps_longitude = ''
+local copy_gps_elevation = ''
+
+local function copy_gps()
+  local sel_images = dt.gui.action_images
+
+  copy_gps_latitude = ''
+  copy_gps_longitude = ''
+  copy_gps_elevation = ''
+
+  for _,image in ipairs(sel_images) do
+    if not image then
+      copy_gps_have_data = false
+    else
+      copy_gps_have_data = true
+      if (image.latitude and label_copy_gps_lat.value) then
+        copy_gps_latitude = image.latitude
+      end
+      if (image.longitude and label_copy_gps_lon.value) then
+          copy_gps_longitude = image.longitude
+      end
+      if (image.elevation and label_copy_gps_ele.value) then
+          copy_gps_elevation = image.elevation
+      end
+    end
+
+    label_copy_gps_lat.label = _("latitude: ") .. copy_gps_latitude
+    label_copy_gps_lon.label = _("longitude: ") ..copy_gps_longitude
+    label_copy_gps_ele.label = _("elevation: ") .. copy_gps_elevation
+
+    return
+  end
+end
+
+local function paste_gps(image)
+  local sel_images = dt.gui.action_images
+
+  for _,image in ipairs(sel_images) do
+    if (label_copy_gps_lat.value) then
+      image.latitude = copy_gps_latitude
+    end
+    if (label_copy_gps_lon.value) then
+      image.longitude = copy_gps_longitude
+    end
+    if (label_copy_gps_ele.value) then
+      image.elevation = copy_gps_elevation
+    end
+  end
+end
+
+local function open_location_in_gnome_maps()
+
+  if not df.check_if_bin_exists("gnome-maps") then
+    dt.print_error(_("gnome-maps not found"))
+    return
+  end
+
+  local sel_images = dt.gui.action_images
+
+  local lat1 = 0;
+  local lon1 = 0;
+  local i = 0;
+
+  -- Use the first image with geo information
+  for _,image in ipairs(sel_images) do
+    if ((image.longitude and image.latitude) and
+        (image.longitude ~= 0 and image.latitude ~= 90) -- Sometimes the north-pole but most likely just wrong data
+       ) then
+        lat1 = string.gsub(image.latitude, ",", ".");
+        lon1 = string.gsub(image.longitude, ",", ".");
+        break
+    end
+  end
+
+  local startCommand
+  startCommand = "gnome-maps \"geo:" .. lat1 .. "," .. lon1 .."\""
+  dt.print_error(startCommand)
+
+  dt.control.execute(startCommand)
+
+end
+
+-- Trim Funktion from: http://lua-users.org/wiki/StringTrim
+local function trim12(s)
+ local from = s:match"^%s*()"
+ return from > #s and "" or s:match(".*%S", from)
+end
+
+local function reverse_geocode()
+
+  if not df.check_if_bin_exists("curl") then
+    dt.print_error(_("curl not found"))
+    return
+  end	
+
+  if not df.check_if_bin_exists("jq") then
+    dt.print_error(_("jq not found"))
+    return
+  end	
+
+  local sel_images = dt.gui.selection() --action_images
+  
+  local lat1 = 0;
+  local lon1 = 0;
+  local i = 0;
+
+  -- Use the first image with geo information
+  for _,image in ipairs(sel_images) do
+    if ((image.longitude and image.latitude) and
+        (image.longitude ~= 0 and image.latitude ~= 90) -- Sometimes the north-pole but most likely just wrong data
+       ) then
+        lat1 = string.gsub(image.latitude, ",", ".");
+        lon1 = string.gsub(image.longitude, ",", ".");
+        break
+      end
+    end
+
+    local startCommand
+    
+    local tokan = dt.preferences.read("geoToolbox","mapBoxKey","string")
+    local types = "country";
+    local types = "region";
+    local types = "place";
+    local types = "poi";
+
+    -- MapBox documentation
+    -- https://www.mapbox.com/api-documentation/#retrieve-places-near-a-location
+    -- curl -s, --silent        Silent mode (don't output anything)
+    -- jq could be replaced with a Lua JSON parser
+    startCommand = string.format("curl --silent \"https://api.mapbox.com/geocoding/v5/mapbox.places/%s,%s.json?types=%s&access_token=%s\" | jq '.features | .[0] | '.text''", lon1, lat1, types, tokan)
+
+    local handle = io.popen(startCommand)
+    local result = trim12(handle:read("*a"))
+    handle:close()
+    
+    -- Errorhandling would be nice
+    --dt.print_error("startCommand: "..startCommand)
+    --dt.print_error("result: '"..result.."'")
+
+    if (result ~= "null") then
+      dt.print(string.sub(result, 2, string.len(result)-2))   
+    end
+
+end
+
+-- I used code from here:
+-- http://stackoverflow.com/questions/27928/how-do-i-calculate-distance-between-two-latitude-longitude-points
+local function get_distance(lat1, lon1, ele1, lat2, lon2, ele2)
+
+    local earthRadius = 6371; -- Radius of the earth in km
+    local dLat = math.rad(lat2-lat1);  -- deg2rad below
+    local dLon = math.rad(lon2-lon1);
+    local a =
+      math.sin(dLat/2) * math.sin(dLat/2) +
+      math.cos(math.rad(lat1)) * math.cos(math.rad(lat2)) *
+      math.sin(dLon/2) * math.sin(dLon/2)
+      ;
+    local angle = 2 * math.atan2(math.sqrt(a), math.sqrt(1-a));
+    local distance = earthRadius * angle; -- Distance in km
+
+    -- Add the elevation to the calculation
+    local elevation = 0
+    elevation = math.abs(ele1 - ele2) / 1000;  --in km
+
+    if (elevation > 0) then
+      distance = math.sqrt(math.pow(elevation,2) + math.pow(distance,2) )
+    end
+    return distance
+end
+
+local function calc_distance()
+
+	  local lat1 = 0;
+    local lon1 = 0;
+    local lat2 = 0;
+    local lon2 = 0;
+    local ele1 = 0;
+    local ele2 = 0;
+    local i = 0;
+
+    local sel_images = dt.gui.selection()
+
+    for _,image in ipairs(sel_images) do
+      if ((image.longitude and image.latitude) and
+            (image.longitude ~= 0 and image.latitude ~= 90) -- Sometimes the north-pole but most likely just wrong data
+           ) then
+
+        i = i + 1;
+
+        if (i == 1) then
+          lat1 = image.latitude
+          lon1 = image.longitude
+          if (image.elevation) then
+            ele1 = image.elevation
+          else
+            ele1 = 0
+          end
+        end
+
+        lat2 = image.latitude
+        lon2 = image.longitude
+        if (image.elevation) then
+          ele2 = image.elevation
+        else
+          ele2 = 0;
+        end
+
+      end
+    end
+
+    local distance = get_distance(lat1, lon1, ele1, lat2, lon2, ele2)
+
+    if (distance < 1) then
+        distance = distance * 1000
+        distanceUnit = _("m")
+    else
+        distanceUnit = _("km")
+    end
+
+    return string.format(_("Distance: %.2f %s"), distance, distanceUnit)
+end
+
+local function print_calc_distance()
+    dt.print(calc_distance())
+end
+
+local function toolbox_calc_distance()
+    labelDistance.label = calc_distance()
+end
+
+local altitude_file_chooser_button = dt.new_widget("file_chooser_button")
+  {
+    title = _("export altitude CSV"),  -- The title of the window when choosing a file
+    value = "",                     -- The currently selected file
+    is_directory = true             -- True if the file chooser button only allows directories to be selecte
+  }
+
+local altitude_filename = dt.new_widget("entry")
+  {
+    text = "altitude.csv",
+    placeholder = "altitude.csv",
+    editable = true,
+    tooltip = _("Name of the exported file"),
+    reset_callback = function(self) self.text = "text" end
+  }
+
+local function altitude_profile()
+	  dt.print(_("Start export"))
+    local sel_images = dt.gui.action_images
+
+    local lat1 = 0;
+    local lon1 = 0;
+    local lat2 = 0;
+    local lon2 = 0;
+    local ele1 = 0;
+    local ele2 = 0;
+    local i = 0;
+    local csv_file = '';
+    csv_file = "km;m".."\n";
+
+    local distance = 0;
+    local distanceFromStart = 0;
+    local elevation = 0;
+    local elevationAdd = 0;
+
+    local sel_images = dt.gui.action_images
+    for _,image in ipairs(sel_images) do
+      if ((not isnan(image.longitude) and not isnan(image.latitude) and not isnan(image.elevation) and image.elevation) and
+            (image.longitude ~= 0 and image.latitude ~= 90) -- Sometimes the north-pole but most likely just wrong data
+           ) then
+
+        i = i + 1;
+
+        if (i == 1) then
+          lat1 = image.latitude
+          lon1 = image.longitude
+          ele1 = image.elevation
+        else
+          lat1 = lat2
+          lon1 = lon2
+          ele1 = ele2
+        end
+
+        lat2 = image.latitude
+        lon2 = image.longitude
+        ele2 = image.elevation
+
+        if (i == 1) then
+          distanceFromStart = 0
+        else
+          local distance = get_distance(lat1, lon1, ele1, lat2, lon2, ele2)
+          distanceFromStart = distanceFromStart + distance;
+        end
+
+        csv_file = csv_file .. distanceFromStart..";"..image.elevation.."\n";
+
+      end
+
+    end --for
+
+    local exportDirectory = altitude_file_chooser_button.value
+    local exportFilename = altitude_filename.text
+    if (exportFilename == '') then
+      exportFilename = altitude_filename.placeholder
+    end
+    file = io.open(exportDirectory.."/"..exportFilename, "w")
+    file:write(csv_file)
+    file:close()
+    dt.print(_("File created in ")..exportDirectory)
+
+end
+
+
+local separator = dt.new_widget("separator"){}
+local separator2 = dt.new_widget("separator"){}
+local separator3 = dt.new_widget("separator"){}
+local separator4 = dt.new_widget("separator"){}
+local separator5 = dt.new_widget("separator"){}
+
+dt.register_lib(
+  "geoToolbox",        -- Module name
+  "geo toolbox",       -- name
+  true,                -- expandable
+  false,               -- resetable
+  {[dt.gui.views.lighttable] = {"DT_UI_CONTAINER_PANEL_RIGHT_CENTER", 100}},   -- containers
+  dt.new_widget("box")
+  {
+    orientation = "vertical",
+    dt.new_widget("button")
+    {
+      label = _("select geo images"),
+      tooltip = _("Select all images with GPS information"),
+      clicked_callback = select_with_gps
+    },
+    dt.new_widget("button")
+    {
+      label = _("select non-geo images"),
+      tooltip = _("Select all images without GPS information"),
+      clicked_callback = select_without_gps
+    },
+    separator,--------------------------------------------------------
+    dt.new_widget("button")
+    {
+      label = _("copy GPS data"),
+      tooltip = _("Copy GPS data"),
+      clicked_callback = copy_gps
+    },
+    label_copy_gps_lat,
+    label_copy_gps_lon,
+    label_copy_gps_ele,
+    dt.new_widget("button")
+    {
+      label = _("paste GPS data"),
+      tooltip = _("Paste GPS data"),
+      clicked_callback = paste_gps
+    },
+    separator2,--------------------------------------------------------
+--ToDo: This need a better UI
+--[[
+    dt.new_widget("button")
+    {
+      label = "get 1st coordinate",
+      tooltip = "Select first image and click this button",
+      clicked_callback = get_first_coordinate
+    },
+    dt.new_widget("button")
+    {
+      label = "get 2nd coordinate",
+      tooltip = "Select second image and click this button",
+      clicked_callback = get_second_coordinate
+    },
+    dt.new_widget("button")
+    {
+      label = "calc in between",
+      tooltip = "Select third image and click this button",
+      clicked_callback = calc_in_between
+    },
+    calc_in_between_slider,
+    separator3,--------------------------------------------------------
+]]
+    dt.new_widget("button")
+    {
+      label = _("open in Gnome Maps"),
+      tooltip = _("Open location in Gnome Maps"),
+      clicked_callback = open_location_in_gnome_maps
+    },
+    separator4,--------------------------------------------------------
+    dt.new_widget("button")
+    {
+      label = _("reverse geocode"),
+      tooltip = _("This just shows the name of the location, but doesn't add it as tag"),
+      clicked_callback = reverse_geocode
+    },
+    separator5,--------------------------------------------------------
+    dt.new_widget("label"){label = _("altitude CSV export")},
+    altitude_file_chooser_button,
+    altitude_filename,
+    dt.new_widget("button")
+    {
+      label = _("export altitude CSV file"),
+      tooltip = _("Create an altitude profile using the GPS data in the metadata"),
+      clicked_callback = altitude_profile
+    },
+    labelDistance
+  },
+  nil,-- view_enter
+  nil -- view_leave
+)
+
+-- Preferences
+dt.preferences.register("geoToolbox",
+	"mapBoxKey",
+	"string",
+	_("geoToolbox export: MapBox Key"),
+	_("https://www.mapbox.com/studio/account/tokens"),
+	'' )
+
+-- Register
+dt.register_event("shortcut", print_calc_distance, _("Calculate the distance from latitude and longitude in km"))
+dt.register_event("mouse-over-image-changed", toolbox_calc_distance)
+
+dt.register_event("shortcut", select_with_gps, _("Select all images with GPS information"))
+dt.register_event("shortcut", select_without_gps, _("Select all images without GPS information"))
+
+-- vim: shiftwidth=2 expandtab tabstop=2 cindent syntax=lua
+-- kate: hl Lua;

contrib/geo_uri.lua

@@ -1,89 +0,0 @@
---[[
-    This file is part of darktable,
-    Copyright 2016 by Tobias Jakobs.
-
-    This program is free software: you can redistribute it and/or modify
-    it under the terms of the GNU General Public License as published by
-    the Free Software Foundation; either version 3 of the License, or
-    (at your option) any later version.
-
-    This program is distributed in the hope that it will be useful,
-    but WITHOUT ANY WARRANTY; without even the implied warranty of
-    MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
-    GNU General Public License for more details.
-
-    You should have received a copy of the GNU General Public License
-    along with this program.  If not, see <http://www.gnu.org/licenses/>.
-]]
---[[
-darktable script to open a geo uri in gnome-maps
-
-ADDITIANAL SOFTWARE NEEDED FOR THIS SCRIPT
-* gnome-maps >= 3.20
-
-USAGE
-* require this script from your main lua file
-* register a shortcut
-]]
-
-local dt = require "darktable"
-require "official/yield"
-
-local gettext = dt.gettext
-dt.configuration.check_version(...,{3,0,0})
-
--- Tell gettext where to find the .mo file translating messages for a particular domain
-gettext.bindtextdomain("geo_uri",dt.configuration.config_dir.."/lua/")
-
-local function _(msgid)
-    return gettext.dgettext("geo_uri", msgid)
-end
-
-local function checkIfBinExists(bin)
-  local handle = io.popen("which "..bin)
-  local result = handle:read()
-  local ret
-  handle:close()
-  if (not result) then
-    dt.print_error(bin.." not found")
-    ret = false
-  end
-  ret = true
-  return ret
-end
-
-local function openLocationInGnomeMaps()
-  if not checkIfBinExists("gnome-maps") then
-    dt.print_error(_("gnome-maps not found"))
-    return
-  end	
-    
-  local sel_images = dt.gui.selection()
-  
-  local lat1 = 0;
-  local lon1 = 0;
-  local i = 0;
-  
-  -- Use the first image with geo information
-  for _,image in ipairs(sel_images) do
-    if ((image.longitude and image.latitude) and 
-        (image.longitude ~= 0 and image.latitude ~= 90) -- Sometimes the north-pole but most likely just wrong data
-       ) then
-        lat1 = image.latitude;
-        lon1 = image.longitude;
-        break
-    end
-
-    local startCommand
-    startCommand = "gnome-maps \"geo:" .. lat1 .. "," .. lon1 .."\""
-    dt.print_error(startCommand)
-    
-    if dt.control.execute( startCommand) then
-      dt.print(_("Command failed ..."))
-    end
-
-  end
-end
-
--- Register
-dt.register_event("shortcut", openLocationInGnomeMaps, _("Open Location in Gnome Maps"))

contrib/gimp.lua

@@ -64,196 +64,56 @@
 ]]
 
 local dt = require "darktable"
-require "official/yield"
-
+local du = require "lib/dtutils"
+local df = require "lib/dtutils.file"
+local dtsys = require "lib/dtutils.system"
 local gettext = dt.gettext
+local gimp_widget = nil
 
-dt.configuration.check_version(...,{3,0,0})
+du.check_min_api_version("5.0.0", "gimp") 
 
 -- Tell gettext where to find the .mo file translating messages for a particular domain
-gettext.bindtextdomain("gimp",dt.configuration.config_dir.."/lua/")
-
-local function split_filepath(str)
-  local result = {}
-  -- Thank you Tobias Jakobs for the awesome regular expression, which I tweaked a little
-  result["path"], result["filename"], result["basename"], result["filetype"] = string.match(str, "(.-)(([^\\/]-)%.?([^%.\\/]*))$")
-  return result
-end
-
-local function get_path(str)
-  local parts = split_filepath(str)
-  return parts["path"]
-end
-
-local function get_filename(str)
-  local parts = split_filepath(str)
-  return parts["filename"]
-end
-
-local function get_basename(str)
-  local parts = split_filepath(str)
-  return parts["basename"]
-end
-
-local function get_filetype(str)
-  local parts = split_filepath(str)
-  return parts["filetype"]
-end
+gettext.bindtextdomain("gimp",dt.configuration.config_dir.."/lua/locale/")
 
 local function _(msgid)
     return gettext.dgettext("gimp", msgid)
 end
 
-local function checkIfBinExists(bin)
-  local handle = io.popen("which "..bin)
-  local result = handle:read()
-  local ret
-  handle:close()
-  if (result) then
-    dt.print_error("true checkIfBinExists: "..bin)
-    ret = true
-  else
-    dt.print_error(bin.." not found")
-    ret = false
-  end
-  return ret
-end
-
--- Thanks Tobias Jakobs for the idea and the correction
-function checkIfFileExists(filepath)
-  local file = io.open(filepath,"r")
-  local ret
-  if file ~= nil then 
-    io.close(file) 
-    dt.print_error("true checkIfFileExists: "..filepath)
-    ret = true
-  else 
-    dt.print_error(filepath.." not found")
-    ret = false
-  end
-  return ret
-end
-
-local function filename_increment(filepath)
-
-  -- break up the filepath into parts
-  local path = get_path(filepath)
-  local basename = get_basename(filepath)
-  local filetype = get_filetype(filepath)
-
-  -- check to see if we've incremented before
-  local increment = string.match(basename, "_(%d-)$")
-
-  if increment then
-    -- we do 2 digit increments so make sure we didn't grab part of the filename
-    if string.len(increment) > 2 then
-      -- we got the filename so set the increment to 01
-      increment = "01"
-    else
-      increment = string.format("%02d", tonumber(increment) + 1)
-      basename = string.gsub(basename, "_(%d-)$", "")
-    end
-  else
-    increment = "01"
-  end
-  local incremented_filepath = path .. basename .. "_" .. increment .. "." .. filetype
-
-  dt.print_error("original file was " .. filepath)
-  dt.print_error("incremented file is " .. incremented_filepath)
-
-  return incremented_filepath
-end
-
-local function groupIfNotMember(img, new_img)
+local function group_if_not_member(img, new_img)
   local image_table = img:get_group_members()
   local is_member = false
   for _,image in ipairs(image_table) do
-    dt.print_error(image.filename .. " is a member")
+    dt.print_log(image.filename .. " is a member")
     if image.filename == new_img.filename then
       is_member = true
-      dt.print_error("Already in group")
+      dt.print_log("Already in group")
     end
   end
   if not is_member then
-    dt.print_error("group leader is "..img.group_leader.filename)
+    dt.print_log("group leader is "..img.group_leader.filename)
     new_img:group_with(img.group_leader)
-    dt.print_error("Added to group")
+    dt.print_log("Added to group")
   end
 end
 
-local function sanitize_filename(filepath)
-  local path = get_path(filepath)
-  local basename = get_basename(filepath)
-  local filetype = get_filetype(filepath)
-
-  local sanitized = string.gsub(basename, " ", "\\ ")
-
-  return path .. sanitized .. "." .. filetype
-end
-
 local function show_status(storage, image, format, filename,
   number, total, high_quality, extra_data)
     dt.print(string.format(_("Export Image %i/%i"), number, total))
 end
 
-local function fileCopy(fromFile, toFile)
-  local result = nil
-  -- if cp exists, use it
-  if checkIfBinExists("cp") then
-    result = os.execute("cp '" .. fromFile .. "' '" .. toFile .. "'")
-  end
-  -- if cp was not present, or if cp failed, then a pure lua solution
-  if not result then
-    local fileIn, err = io.open(fromFile, 'rb')
-    if fileIn then
-      local fileOut, errr = io.open(toFile, 'w')
-      if fileOut then
-        local content = fileIn:read(4096)
-        while content do
-          fileOut:write(content)
-          content = fileIn:read(4096)
-        end
-        result = true
-        fileIn:close()
-        fileOut:close()
-      else
-        dt.print_error("fileCopy Error: " .. errr)
-      end
-    else
-      dt.print_error("fileCopy Error: " .. err)
-    end
-  end
-  return result
-end
-
-local function fileMove(fromFile, toFile)
-  local success = os.rename(fromFile, toFile)
-  if not success then
-    -- an error occurred, so let's try using the operating system function
-    if checkIfBinExists("mv") then
-      success = os.execute("mv '" .. fromFile .. "' '" .. toFile .. "'")
-    end
-    -- if the mv didn't exist or succeed, then...
-    if not success then
-      -- pure lua solution
-      success = fileCopy(fromFile, toFile)
-      if success then
-        os.remove(fromFile)
-      else
-        dt.print_error("fileMove Error: Unable to move " .. fromFile .. " to " .. toFile .. ".  Leaving " .. fromFile .. " in place.")
-        dt.print(string.format(_("Unable to move edited file into collection. Leaving it as %s"), fromFile))
-      end
-    end
-  end
-  return success  -- nil on error, some value if success
-end
-
 local function gimp_edit(storage, image_table, extra_data) --finalize
-  if not checkIfBinExists("gimp") then
+
+  local gimp_executable = df.check_if_bin_exists("gimp")
+
+  if not gimp_executable then
     dt.print_error(_("GIMP not found"))
     return
   end
 
+  if dt.configuration.running_os == "macos" then
+    gimp_executable = "open -W -a " .. gimp_executable
+  end
+
   -- list of exported images
   local img_list
 
@@ -261,18 +121,18 @@ local function gimp_edit(storage, image_table, extra_data) --finalize
   img_list = ""
 
   for _,exp_img in pairs(image_table) do
-    exp_img = sanitize_filename(exp_img)
+    exp_img = df.sanitize_filename(exp_img)
     img_list = img_list ..exp_img.. " "
   end
 
   dt.print(_("Launching GIMP..."))
 
   local gimpStartCommand
-  gimpStartCommand = "gimp "..img_list
+  gimpStartCommand = gimp_executable .. " " .. img_list
 
-  dt.print_error(gimpStartCommand)
+  dt.print_log(gimpStartCommand)
 
-  dt.control.execute( gimpStartCommand)
+  dtsys.external_command(gimpStartCommand)
 
   -- for each of the image, exported image pairs
   --   move the exported image into the directory with the original
@@ -281,28 +141,28 @@ local function gimp_edit(storage, image_table, extra_data) --finalize
 
   for image,exported_image in pairs(image_table) do
 
-    local myimage_name = image.path .. "/" .. get_filename(exported_image)
+    local myimage_name = image.path .. "/" .. df.get_filename(exported_image)
 
-    while checkIfFileExists(myimage_name) do
-      myimage_name = filename_increment(myimage_name)
+    while df.check_if_file_exists(myimage_name) do
+      myimage_name = df.filename_increment(myimage_name)
       -- limit to 99 more exports of the original export
-      if string.match(get_basename(myimage_name), "_(d-)$") == "99" then 
+      if string.match(df.get_basename(myimage_name), "_(d-)$") == "99" then
         break
       end
     end
 
-    dt.print_error("moving " .. exported_image .. " to " .. myimage_name)
-    local result = fileMove(exported_image, myimage_name)
+    dt.print_log("moving " .. exported_image .. " to " .. myimage_name)
+    local result = df.file_move(exported_image, myimage_name)
 
     if result then
-      dt.print_error("importing file")
+      dt.print_log("importing file")
       local myimage = dt.database.import(myimage_name)
 
-      groupIfNotMember(image, myimage)
+      group_if_not_member(image, myimage)
 
       for _,tag in pairs(dt.tags.get_tags(image)) do
         if not (string.sub(tag.name,1,9) == "darktable") then
-          dt.print_error("attaching tag")
+          dt.print_log("attaching tag")
           dt.tags.attach(tag,myimage)
         end
       end
@@ -312,8 +172,13 @@ local function gimp_edit(storage, image_table, extra_data) --finalize
 end
 
 -- Register
-dt.register_storage("module_gimp", _("Edit with GIMP"), show_status, gimp_edit)
+
+local executables = {"gimp"}
+
+if dt.configuration.running_os ~= "linux" then
+  gimp_widget = df.executable_path_widget(executables)
+end
+
+dt.register_storage("module_gimp", _("Edit with GIMP"), show_status, gimp_edit, nil, nil, gimp_widget)
 
 --
-
-

contrib/gps_select.lua

@@ -1,60 +0,0 @@
---[[
-    This file is part of darktable,
-    Copyright 2014 by Tobias Jakobs.
-
-    This program is free software: you can redistribute it and/or modify
-    it under the terms of the GNU General Public License as published by
-    the Free Software Foundation; either version 3 of the License, or
-    (at your option) any later version.
-
-    This program is distributed in the hope that it will be useful,
-    but WITHOUT ANY WARRANTY; without even the implied warranty of
-    MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
-    GNU General Public License for more details.
-
-    You should have received a copy of the GNU General Public License
-    along with this program.  If not, see <http://www.gnu.org/licenses/>.
-]]
---[[
-darktable select images with or without GPS informations
-
-USAGE
-* require this script from your main lua file
-* register a shortcuts
-]]
-   
-local dt = require "darktable"
-local gettext = dt.gettext
-dt.configuration.check_version(...,{2,0,1},{3,0,0})
-
--- Tell gettext where to find the .mo file translating messages for a particular domain
-gettext.bindtextdomain("gps_select",dt.configuration.config_dir.."/lua/")
-
-local function _(msgid)
-    return gettext.dgettext("gps_select", msgid)
-end
-
-table = require "table"
-
-local function selectWithGPS()
-   local selection = {}
-   for _,image in ipairs(dt.database) do
-      if (image.longitude and image.latitude) then
-         table.insert(selection,image)
-      end
-   end
-   dt.gui.selection(selection)
-end
-
-local function selectWithoutGPS()
-   local selection = {}
-   for _,image in ipairs(dt.database) do
-      if (not image.longitude and not image.latitude) then
-         table.insert(selection,image)
-      end
-   end
-   dt.gui.selection(selection)
-end
-
-dt.register_event("shortcut", selectWithGPS, _("Select all images with GPS information"))
-dt.register_event("shortcut", selectWithoutGPS, _("Select all images without GPS information"))

contrib/gpx_export.lua

@@ -0,0 +1,157 @@
+--[[
+    This file is part of darktable,
+    copyright (c) 2017 Jannis_V
+
+    darktable is free software: you can redistribute it and/or modify
+    it under the terms of the GNU General Public License as published by
+    the Free Software Foundation, either version 3 of the License, or
+    (at your option) any later version.
+    darktable is distributed in the hope that it will be useful,
+     but WITHOUT ANY WARRANTY; without even the implied warranty of
+    MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+    GNU General Public License for more details.
+    You should have received a copy of the GNU General Public License
+    along with darktable.  If not, see <http://www.gnu.org/licenses/>.
+]]
+
+--[[
+Simple darktable GPX generator script
+
+This script generates a GPX track from all images having GPS latitude
+and longitude information.
+For each source folder, a separate <trk> is generated in the gpx file.
+]]
+
+local dt = require "darktable"
+local df = require "lib/dtutils.file"
+local dl = require "lib/dtutils"
+local gettext = dt.gettext
+
+dl.check_min_api_version("3.0.0", "gpx-export") 
+
+-- Tell gettext where to find the .mo file translating messages for a particular domain
+gettext.bindtextdomain("gpx_export",dt.configuration.config_dir.."/lua/locale/")
+
+local function _(msgid)
+  return gettext.dgettext("gpx_export", msgid)
+end
+
+local path_entry = dt.new_widget("entry")
+{
+  text = dt.preferences.read("gpx_exporter", "gpxExportPath", "string"),
+  editable=true,
+  reset_callback = function(self)
+    self.text = "~/darktable.gpx"
+    dt.preferences.write("gpx_exporter", "gpxExportPath", "string", self.text)
+  end,
+  tooltip = _("gpx file path"),
+}
+
+local function stop_job(job)
+  job.valid = false
+end
+
+local function create_gpx_file()
+  dt.preferences.write("gpx_exporter", "gpxExportPath", "string", path_entry.text)
+
+  path = path_entry.text:gsub("^~", os.getenv("HOME")) -- Expand ~ to home
+  path = path:gsub("//", "/")
+
+  dt.print(_("exporting gpx file..."))
+
+  job = dt.gui.create_job(_("gpx export"), true, stop_job)
+
+  local sel_images = dt.gui.action_images
+  local segments = {}
+  for key,image in dl.spairs(sel_images, function(t, a, b) return t[b].path > t[a].path end) do
+
+    print(image.path)
+
+    if (job.valid) then
+      job.percent = (key - 1) / #sel_images
+
+      if ((image.longitude and image.latitude) and
+        (image.longitude ~= 0 and image.latitude ~= 90) -- Just in case
+      ) then
+        if (segments[image.path] == nil) then
+          segments[image.path] = {}
+        end
+
+        if (image.exif_datetime_taken == "") then
+          dt.print(image.path.."/"..image.filename.._(" does not have date information and won't be processed"))
+          print(image.path.."/"..image.filename.._(" does not have date information and won't be processed")) -- Also print to terminal
+        else
+          segments[image.path][image.filename] = image
+        end
+      end
+    else
+      break
+    end
+  end
+
+  local gpx_file = "<?xml version=\"1.0\" encoding=\"UTF-8\" standalone=\"no\" ?>\n"
+  gpx_file = gpx_file.."<gpx xmlns=\"http://www.topografix.com/GPX/1/1\" creator=\"Darktable GPX Exporter\"\n"
+  gpx_file = gpx_file.."version=\"1.1\" xmlns:xsi=\"http://www.w3.org/2001/XMLSchema-instance\"\n"
+  gpx_file = gpx_file.."xsi:schemaLocation=\"http://www.topografix.com/GPX/1/1 http://www.topografix.com/GPX/1/1/gpx.xsd\">\n"
+
+  for key, folder in dl.spairs(segments) do
+    gpx_file = gpx_file.."\t<trk>\n"
+    gpx_file = gpx_file.."\t\t<name>"..key.."</name>\n";
+    gpx_file = gpx_file.."\t\t<trkseg>\n";
+    for key2, image in dl.spairs(folder, function(t, a, b) return t[b].exif_datetime_taken > t[a].exif_datetime_taken end) do
+      date_format = "(%d+):(%d+):(%d+) (%d+):(%d+):(%d+)"
+      my_year, my_month, my_day, my_hour, my_min, my_sec = image.exif_datetime_taken:match(date_format)
+
+      local my_timestamp = os.time({year=my_year, month=my_month, day=my_day, hour=my_hour, min=my_min, sec=my_sec})
+
+      gpx_file = gpx_file.."\t\t\t<trkpt lat=\""..string.gsub(tostring(image.latitude), ",", ".").."\" lon=\""..string.gsub(tostring(image.longitude), ",", ".").."\">\n"
+      gpx_file = gpx_file.."\t\t\t\t<time>"..os.date("!%Y-%m-%dT%H:%M:%SZ", my_timestamp).."</time>\n"
+      gpx_file = gpx_file.."\t\t\t\t<name>"..image.path.."/"..image.filename.."</name>\n"
+      gpx_file = gpx_file.."\t\t\t</trkpt>\n"
+    end
+    gpx_file = gpx_file.."\t\t</trkseg>\n";
+    gpx_file = gpx_file.."\t</trk>\n";
+  end
+
+  job.valid = false
+
+  gpx_file = gpx_file.."</gpx>\n";
+
+  local file = io.open(path, "w")
+  if (file == nil) then
+    dt.print(_("invalid path: ")..path)
+  else
+    file:write(gpx_file)
+    file:close()
+    dt.print(_("gpx file created: ")..path)
+  end
+end
+
+dt.register_lib(
+  "gpx_exporter",
+  "gpx export",
+  true, -- expandable
+  true, -- resetable
+  {[dt.gui.views.lighttable] = {"DT_UI_CONTAINER_PANEL_RIGHT_CENTER", 100}},   -- containers
+  dt.new_widget("box")
+  {
+    orientation = "vertical",
+    dt.new_widget("button")
+    {
+      label = _("export"),
+      tooltip = _("export gpx file"),
+      clicked_callback = create_gpx_file
+    },
+    dt.new_widget("box")
+    {
+      orientation = "horizontal",
+      dt.new_widget("label")
+      {
+        label = _("file:"),
+      },
+      path_entry
+    },
+  },
+  nil,-- view_enter
+  nil -- view_leave
+)

contrib/hugin.lua

@@ -28,102 +28,203 @@ ADDITIONAL SOFTWARE NEEDED FOR THIS SCRIPT
 * hugin
 
 USAGE
-* require this file from your main luarc config file.
+* require this file from your main luarc config file
+* set the hugin tool paths (on some platforms)
+* if hugin gui mode is used, save the final result in the tmp directory with the first file name and _pano as suffix for the image to be automatically imported to DT afterwards
 
 This plugin will add a new storage option and calls hugin after export.
 ]]
 
 local dt = require "darktable"
+local du = require "lib/dtutils"
+local df = require "lib/dtutils.file"
+local log = require "lib/dtutils.log"
+local dtsys = require "lib/dtutils.system"
 require "official/yield"
-
 local gettext = dt.gettext
 
--- works with darktable API version 2.0.0 and 3.0.0
-dt.configuration.check_version(...,{2,0,0},{3,0,0})
+local namespace = 'module_hugin'
+local user_pref_str = 'prefer_gui'
+local user_prefer_gui = dt.preferences.read(namespace, user_pref_str, "bool")
+log.msg(log.info, "user_prefer_gui set to ", user_prefer_gui)
+local hugin_widget = nil
+local exec_widget = nil
+local executable_table = {"hugin", "hugin_executor", "pto_gen"}
+
+-- get the proper path quoting quote
+local PQ = dt.configuration.running_os == "windows" and '"' or "'"
+
+-- works with darktable API version from 5.0.0 on
+du.check_min_api_version("5.0.0", "hugin") 
 
 -- Tell gettext where to find the .mo file translating messages for a particular domain
-gettext.bindtextdomain("hugin",dt.configuration.config_dir.."/lua/")
+gettext.bindtextdomain("hugin",dt.configuration.config_dir.."/lua/locale/")
 
 local function _(msgid)
-    return gettext.dgettext("hugin", msgid)
+  return gettext.dgettext("hugin", msgid)
 end
 
-local function checkIfBinExists(bin)
-  local handle = io.popen("which "..bin)
-  local result = handle:read()
-  local ret
-  handle:close()
-  if (result) then
-    dt.print_error("true checkIfBinExists: "..bin)
-    ret = true
-  else
-    dt.print_error(bin.." not found")
-    ret = false
-  end
-
-
-  return ret
+local function user_preference_changed(widget)
+  user_prefer_gui = widget.value
+  dt.preferences.write(namespace, user_pref_str, "bool", user_prefer_gui)
 end
 
 local function show_status(storage, image, format, filename,
   number, total, high_quality, extra_data)
-  dt.print("Export to Hugin "..tostring(number).."/"..tostring(total))
+  dt.print("exporting to hugin: "..tostring(number).."/"..tostring(total))
 end
 
 local function create_panorama(storage, image_table, extra_data) --finalize
-  if not checkIfBinExists("hugin") then
-    dt.print_error(_("hugin not found"))
-    return
-  end
-
 -- Since Hugin 2015.0.0 hugin provides a command line tool to start the assistant
 -- http://wiki.panotools.org/Hugin_executor
 -- We need pto_gen to create pto file for hugin_executor
 -- http://hugin.sourceforge.net/docs/manual/Pto_gen.html
 
-  local hugin_executor = false
-  if (checkIfBinExists("hugin_executor") and checkIfBinExists("pto_gen")) then
-    hugin_executor = true
+  -- save the current log level and set log level to
+  --   log.error for normal operations
+  --   log.info for more insight about what is happening
+  --   log.debug for even more information
+
+  local saved_log_level = log.log_level()
+  log.log_level(log.error)
+
+  local hugin = df.check_if_bin_exists("hugin")
+  log.msg(log.info, "hugin set to ", hugin)
+  local hugin_executor = df.check_if_bin_exists("hugin_executor")
+  log.msg(log.info, "hugin_executor set to ", hugin_executor)
+  local pto_gen = df.check_if_bin_exists("pto_gen")
+  log.msg(log.info, "pto_gen set to ", pto_gen)
+
+  local gui_available = false
+  if hugin then
+    gui_available = true
+  else
+    dt.print(_("hugin is not found, did you set the path?"))
+    log.msg(log.error, "hugin executable not found.  Check if the executable is installed.")
+    log.msg(log.error, "If the executable is installed, check that the path is set correctly.")
+    return
+  end
+
+  local cmd_line_available = false
+  if hugin_executor and pto_gen then
+    cmd_line_available = true
   end
 
   -- list of exported images
-  local img_list
+  local img_list = ""
+  local img_set = {}
 
   -- reset and create image list
-  img_list = ""
-
-  for _,v in pairs(image_table) do
-    img_list = img_list ..v.. " "
+  for k,v in pairs(image_table) do
+    log.msg(log.debug, "k is ", k, " and v is ", v)
+    img_list = img_list .. PQ .. v .. PQ .. ' '  -- surround the filename with single quotes to handle spaces
+    table.insert(img_set, k)
   end
 
-  dt.print(_("Will try to stitch now"))
+  log.msg(log.info, "img_list is ", img_list)
 
-  local huginStartCommand
-  if (hugin_executor) then
-    huginStartCommand = "pto_gen "..img_list.." -o "..dt.configuration.tmp_dir.."/project.pto"
-    dt.print(_("Creating pto file"))
-    dt.control.execute( huginStartCommand)
+  -- use first file as basename for output file
+  table.sort(img_set, function(a,b) return a.filename<b.filename end);
+  local first_file
+  for _, k in ipairs(img_set) do
+    first_file = k 
+    break
+  end
 
-    dt.print(_("Running Assistent"))
-    huginStartCommand = "hugin_executor --assistant "..dt.configuration.tmp_dir.."/project.pto"
-    dt.control.execute( huginStartCommand)
+  if first_file == nil then
+    dt.print("no file selected")
+    return
+  end
 
-    huginStartCommand = "hugin "..dt.configuration.tmp_dir.."/project.pto"
+  local pto_path = dt.configuration.tmp_dir..'/project.pto'
+  log.msg(log.info, "pto_path is ", pto_path)
+  local filepath = df.split_filepath(first_file.filename)
+  log.msg(log.info, "filepath is ", filepath)
+  local tmp_filename = filepath['basename'].."_pano.tif"
+  log.msg(log.info, "tmp_filename is ", tmp_filename)
+-- some versions of hugin_executor don't seem to output the result
+-- to the --prefix location, so assume we are working in the tmp
+-- directory and then move the file to the original file location afterwards
+  local src_path = dt.configuration.tmp_dir.."/"..tmp_filename
+  log.msg(log.info, "src_path is ", src_path)
+  local dst_path = first_file.path..'/'..tmp_filename
+  log.msg(log.info, "dst_path is ", dst_path)
+
+  if df.check_if_file_exists(dst_path) then
+    dst_path = df.create_unique_filename(dst_path)
+    log.msg(log.info, "dst_path updated to ", dts_path)
+  end
+
+  dt.print(_("will try to stitch now"))
+  local huginStartCommand = nil
+  if (cmd_line_available and not user_prefer_gui) then
+    log.msg(log.info, "using the command line tools to create panorama")
+    huginStartCommand = pto_gen.." "..img_list.." -o "..pto_path
+    dt.print(_("creating pto file"))
+    log.msg(log.info, "pto creation command is ", huginStartCommand)
+    dtsys.external_command(huginStartCommand)
+
+    dt.print(_("running assistant"))
+    huginStartCommand = hugin_executor.." --assistant "..pto_path
+    dtsys.external_command(huginStartCommand)
+
+    huginStartCommand = hugin_executor..' --stitching --prefix=' .. "'" .. src_path .. "'" .. ' ' .. pto_path
+    log.msg(log.info, "command line huginStartCommand is ", huginStartCommand)
+  elseif gui_available then
+    if (user_prefer_gui) then
+      dt.print(_("launching hugin"))
+    else
+      dt.print(_("unable to find command line tools, launching hugin"))
+    end
+    -- the gui produces a differnt output filename than the command line
+    src_path = dt.configuration.tmp_dir .. "/" .. df.get_basename(img_set[1].filename) .. " - " .. df.get_basename(img_set[#img_set].filename) .. ".tif"
+    log.msg(log.info, "set src_path to ", src_path)
+    huginStartCommand = hugin..' '..img_list
+    log.msg(log.info, "gui huginStartCommand is ", huginStartCommand)
   else
-    huginStartCommand = "hugin "..img_list
+    dt.print(_("hugin isn't available."))
   end
 
-  dt.print_error(huginStartCommand)
-
-  if dt.control.execute( huginStartCommand)
-    then
-    dt.print(_("Command hugin failed ..."))
+  if not (huginStartCommand==nil) then
+    if not dtsys.external_command(huginStartCommand) then
+      dt.print(_("hugin failed ..."))
+      log.msg(log.error, huginStartCommand, " failed")
+    else
+      if df.check_if_file_exists(src_path) then
+        log.msg(log.debug, "found ", src_path, " importing to ", dst_path)
+        df.file_move(src_path, dst_path)
+        dt.print(_("importing file "..dst_path))
+        dt.database.import(dst_path)
+      end
+    end
+    log.log_level(saved_log_level)
   end
 
+  -- cleanup the temp files
+  for k,v in pairs(image_table) do
+    os.remove(v)
+  end
 end
 
 -- Register
-dt.register_storage("module_hugin", _("Hugin Panorama"), show_status, create_panorama)
+if dt.configuration.running_os ~= "linux" then
+  exec_widget = df.executable_path_widget(executable_table)
+end
+
+hugin_widget = dt.new_widget("box") {
+  orientation = "vertical",
+  dt.new_widget("check_button")
+  {
+    label = _("  launch hugin gui"),
+    value = user_prefer_gui,
+    tooltip = _('launch hugin in gui mode'),
+    clicked_callback = user_preference_changed
+  },
+  exec_widget
+}
+
+dt.register_storage(namespace, _("hugin panorama"), show_status, create_panorama, nil, nil, hugin_widget)
+
 
 --
 -- vim: shiftwidth=2 expandtab tabstop=2 cindent syntax=lua

contrib/image_stack.lua

@@ -0,0 +1,587 @@
+--[[
+
+    image_stack.lua - process a stack of images
+
+    Copyright (C) 2018 Bill Ferguson <wpferguson@gmail.com>.
+    Copyright (C) 2016,2017 Holger Klemm 
+
+    This program is free software: you can redistribute it and/or modify
+    it under the terms of the GNU General Public License as published by
+    the Free Software Foundation; either version 3 of the License, or
+    (at your option) any later version.
+
+    This program is distributed in the hope that it will be useful,
+    but WITHOUT ANY WARRANTY; without even the implied warranty of
+    MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+    GNU General Public License for more details.
+
+    You should have received a copy of the GNU General Public License
+    along with this program.  If not, see <http://www.gnu.org/licenses/>.
+]]
+--[[
+    image_stack - export a stack of images and process them, returning the result
+
+    This script provides another storage (export target) for darktable.  Selected
+    images are exported in the specified format to temporary storage.  The images are aligned
+    if the user requests it. When the images are ready, imagemagick is launched and uses
+    the selected evaluate-sequence operator to process the images.  The output file is written
+    to a filename representing the imput files in the format specified by the user.  The resulting 
+    image is imported into the film roll.  The source images can be tagged as part of the file 
+    creation so that  a user can later find the contributing images.
+
+    ADDITIONAL SOFTWARE NEEDED FOR THIS SCRIPT
+    * align_image_stack - http://www.hugin.org
+    * imagemagick - http://www.imagemagick.org
+
+    USAGE
+    * require this script from your main lua file
+    * select the images to process with image_stack
+    * in the export dialog select "image stack" and select the format and bit depth for the
+      exported image
+    * Select whether the images need to be aligned.
+    * Select the stack operator
+    * Select the output format
+    * Select whether to tag the source images used to create the resulting file
+    * Specify executable locations if necessary
+    * Press "export"
+    * The resulting image will be imported
+
+    NOTES
+    Mean is a fairly quick operation.  On my machine (i7-6800K, 16G) it takes a few seconds.  Median, on the other hand
+    takes approximately 10x longer to complete.  Processing 10 and 12 image stacks took over a minute.  I didn't test all
+    the other functions, but the ones I did fell between Mean and Median performance wise.
+
+    BUGS, COMMENTS, SUGGESTIONS
+    * Send to Bill Ferguson, wpferguson@gmail.com
+
+    CHANGES
+
+    THANKS
+    * Thanks to Pat David and his blog entry on blending images, https://patdavid.net/2013/05/noise-removal-in-photos-with-median_6.html
+]]
+
+local dt = require "darktable"
+local du = require "lib/dtutils"
+local df = require "lib/dtutils.file"
+local dtsys = require "lib/dtutils.system"
+local gettext = dt.gettext
+local job = nil
+
+-- path separator constant
+local PS = dt.configuration.running_os == "windows" and "\\" or "/"
+
+-- works with LUA API version 5.0.0
+du.check_min_api_version("5.0.0", "image_stack") 
+
+-- Tell gettext where to find the .mo file translating messages for a particular domain
+gettext.bindtextdomain("image_stack",dt.configuration.config_dir.."/lua/locale/")
+
+local function _(msgid)
+    return gettext.dgettext("image_stack", msgid)
+end
+
+-- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - 
+--  GUI definitions
+-- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - 
+
+local label_align_options= dt.new_widget("section_label"){
+  label = _('image align options')
+}
+
+local label_image_stack_options = dt.new_widget("section_label"){
+  label = _("image stack options")
+}
+
+local label_executable_locations = dt.new_widget("section_label"){
+  label = _("executable locations")
+}
+
+if not(dt.preferences.read("align_image_stack",  "initialized", "bool")) then
+   dt.preferences.write("align_image_stack", "def_radial_distortion", "bool", false)      
+   dt.preferences.write("align_image_stack", "def_optimize_field", "bool", false) 
+   dt.preferences.write("align_image_stack", "def_optimize_image_center", "bool", true) 
+   dt.preferences.write("align_image_stack", "def_auto_crop", "bool", true) 
+   dt.preferences.write("align_image_stack", "def_distortion", "bool", true) 
+   dt.preferences.write("align_image_stack", "def_grid_size", "integer", 5)
+   dt.preferences.write("align_image_stack", "def_control_points", "integer", 8)
+   dt.preferences.write("align_image_stack", "def_control_points_remove", "integer", 3)
+   dt.preferences.write("align_image_stack", "def_correlation", "integer", 9)
+   dt.preferences.write("align_image_stack",  "initialized", "bool", true) 
+end
+
+if not(dt.preferences.read("image_stack", "initialized", "bool")) then
+  dt.preferences.write("image_stack", "align_images", "bool", true)
+  dt.preferences.write("image_stack", "stack_function", "integer", 2)
+  dt.preferences.write("image_stack", "output_format", "integer", 6)
+  dt.preferences.write("image_stack", "tag_images", "bool", true)
+  dt.preferences.write("image_stack", "initialized", "bool", true)
+end
+
+local chkbtn_will_align = dt.new_widget("check_button"){
+  label = _('perform image alignment'),
+  value = dt.preferences.read("image_stack", "align_images", "bool"),
+  tooltip = _('align the image stack before processing')
+}
+  
+local chkbtn_radial_distortion = dt.new_widget("check_button"){
+  label = _('optimize radial distortion for all images'),
+  value = dt.preferences.read("align_image_stack", "def_radial_distortion", "bool"),
+  tooltip = _('optimize radial distortion for all images, \nexcept for first'),
+}
+
+local chkbtn_optimize_field = dt.new_widget("check_button"){
+    label = _('optimize field of view for all images'), 
+    value = dt.preferences.read("align_image_stack", "def_optimize_field", "bool"),
+    tooltip =_('optimize field of view for all images, except for first. \nUseful for aligning focus stacks (DFF) with slightly \ndifferent magnification.'), 
+}
+
+local chkbtn_optimize_image_center = dt.new_widget("check_button"){
+    label = _('optimize image center shift for all images'), 
+    value = dt.preferences.read("align_image_stack", "def_optimize_image_center", "bool"),
+    tooltip =_('optimize image center shift for all images, \nexcept for first.'),   
+}
+
+local chkbtn_auto_crop = dt.new_widget("check_button"){
+    label = _('auto crop the image'), 
+    value = dt.preferences.read("align_image_stack", "def_auto_crop", "bool"),
+    tooltip =_('auto crop the image to the area covered by all images.'),   
+}
+
+local chkbtn_distortion = dt.new_widget("check_button"){
+    label = _('load distortion from lens database'), 
+    value = dt.preferences.read("align_image_stack", "def_distortion", "bool"),
+    tooltip =_('try to load distortion information from lens database'),   
+}
+
+local cmbx_grid_size = dt.new_widget("combobox"){
+    label = _('image grid size'), 
+    tooltip =_('break image into a rectangular grid \nand attempt to find num control points in each section.\ndefault: (5x5)'),
+    value = dt.preferences.read("align_image_stack", "def_grid_size", "integer"), --5
+    "1", "2", "3","4","5","6","7","8","9",
+    reset_callback = function(self)
+       self.value = dt.preferences.read("align_image_stack", "def_grid_size", "integer")
+    end
+} 
+
+local cmbx_control_points = dt.new_widget("combobox"){
+    label = _('control points/grid'), 
+    tooltip =_('number of control points (per grid, see option -g) \nto create between adjacent images \ndefault: (8).'),
+    value = dt.preferences.read("align_image_stack", "def_control_points", "integer"),   --8, "1", "2", "3","4","5","6","7","8","9",
+    "1", "2", "3","4","5","6","7","8","9",             
+    reset_callback = function(self)
+       self.value = dt.preferences.read("align_image_stack", "def_control_points", "integer")
+    end
+} 
+
+local cmbx_control_points_remove = dt.new_widget("combobox"){
+    label = _('remove control points with error'), 
+    tooltip =_('remove all control points with an error higher \nthan num pixels \ndefault: (3)'),
+    value = dt.preferences.read("align_image_stack", "def_control_points_remove", "integer"), --3, "1", "2", "3","4","5","6","7","8","9",
+    "1", "2", "3","4","5","6","7","8","9",              
+    reset_callback = function(self)
+       self.value = dt.preferences.read("align_image_stack", "def_control_points_remove", "integer")
+    end
+} 
+
+local cmbx_correlation  = dt.new_widget("combobox"){
+    label = _('correlation threshold for control points'), 
+    tooltip =_('correlation threshold for identifying \ncontrol points \ndefault: (0.9).'),
+    value = dt.preferences.read("align_image_stack", "def_correlation", "integer"), --9, "0,1", "0,2", "0,3","0,4","0,5","0,6","0,7","0,8","0,9",
+    "0.1", "0.2", "0.3","0.4","0.5","0.6","0.7","0.8","0.9","1.0",     
+    reset_callback = function(self)
+       self.value = dt.preferences.read("align_image_stack", "def_correlation", "integer")
+    end
+} 
+
+local cmbx_stack_function = dt.new_widget("combobox"){
+  label = _("select stack function"),
+  tooltip = _("select function to be \napplied to image stack"),
+  value = dt.preferences.read("image_stack", "stack_function", "integer"), "Mean", "Median", "Abs", "Add", "And", "Divide", "Max", "Min",
+                                                                          "Or", "Subtract", "Sum", "Xor",
+  reset_callback = function(self)
+    self.value = dt.preferences.read("image_stack", "stack_function", "integer")
+  end
+}
+
+local cmbx_output_format = dt.new_widget("combobox"){
+  label = _("select output format"),
+  tooltip = _("choose the format for the resulting image"),
+  value = dt.preferences.read("image_stack", "output_format", "integer"), "EXR", "JPG", "PNG", "PNM", "PPM", "TIF",
+  reset_callback = function(self)
+    self.value = dt.preferences.read("image_stack", "output_format", "integer")
+  end
+}
+
+local chkbtn_tag_source_file = dt.new_widget("check_button"){
+  label = _("tag source images used?"),
+  value = dt.preferences.read("image_stack", "tag_images", "bool"),
+  tooltip = _("tag the source images used to create the output file?")
+}
+
+local image_stack_widget = dt.new_widget("box"){
+  orientation = "vertical",
+  label_align_options,
+  chkbtn_will_align,
+  chkbtn_radial_distortion,
+  chkbtn_optimize_field,
+  chkbtn_optimize_image_center,
+  chkbtn_auto_crop,
+  chkbtn_distortion,
+  cmbx_grid_size,
+  cmbx_control_points,
+  cmbx_control_points_remove,
+  cmbx_correlation,
+  dt.new_widget("separator"){},
+  dt.new_widget("separator"){},
+  label_image_stack_options,
+  cmbx_stack_function,
+  cmbx_output_format,
+  chkbtn_tag_source_file,
+}
+
+local executables = {"align_image_stack", "convert"}
+
+if dt.configuration.running_os ~= "linux" then
+  image_stack_widget[#image_stack_widget + 1] = df.executable_path_widget(executables)
+end
+
+-- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - 
+--  local functions
+-- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - 
+
+local function show_status(storage, image, format, filename, number, total, high_quality, extra_data)
+    dt.print(string.format(_("Export Image %i/%i"), number, total))
+end
+
+-- read the gui and populate the align_image_stack arguments
+
+local function get_align_image_stack_arguments()
+  
+  --Setup Align Image Stack Arguments--
+  local align_args = ""
+  if (chkbtn_radial_distortion.value) then align_args = align_args .. " -d" end
+  if (chkbtn_optimize_field.value) then align_args = align_args .. " -m" end  
+  if (chkbtn_optimize_image_center.value) then align_args = align_args .. " -i" end
+  if (chkbtn_auto_crop.value) then align_args = align_args .. " -C" end     
+  if (chkbtn_distortion.value) then align_args = align_args .. " --distortion" end
+
+  dt.preferences.write("align_image_stack", "def_radial_distortion", "bool", chkbtn_radial_distortion.value)      
+  dt.preferences.write("align_image_stack", "def_optimize_field", "bool", chkbtn_optimize_field.value) 
+  dt.preferences.write("align_image_stack", "def_optimize_image_center", "bool", chkbtn_optimize_image_center.value) 
+  dt.preferences.write("align_image_stack", "def_auto_crop", "bool", chkbtn_auto_crop.value) 
+  dt.preferences.write("align_image_stack", "def_distortion", "bool", chkbtn_distortion.value) 
+
+  align_args = align_args.." -g "..cmbx_grid_size.value
+  align_args = align_args.." -c "..cmbx_control_points.value          
+  align_args = align_args.." -t "..cmbx_control_points_remove.value
+  align_args = align_args.." --corr="..cmbx_correlation.value
+
+  dt.preferences.write("align_image_stack", "def_grid_size", "integer", cmbx_grid_size.selected)
+  dt.preferences.write("align_image_stack", "def_control_points", "integer", cmbx_control_points.selected)
+  dt.preferences.write("align_image_stack", "def_control_points_remove", "integer", cmbx_control_points_remove.selected)
+  dt.preferences.write("align_image_stack", "def_correlation", "integer", cmbx_correlation.selected)
+
+  if (dt.preferences.read("align_image_stack", "align_use_gpu", "bool")) then align_args = align_args .. " --gpu" end
+
+  align_args = align_args .. " -a " .. dt.configuration.tmp_dir .. "/aligned_ "
+
+  return align_args
+end
+
+-- extract, and sanitize, an image list from the supplied image table
+
+local function extract_image_list(image_table)
+  local img_list = ""
+  local result = {}
+
+  for img,expimg in pairs(image_table) do
+    table.insert(result, expimg)
+  end
+  table.sort(result)
+  for _,exp_img in ipairs(result) do
+    img_list = img_list .. " " .. df.sanitize_filename(exp_img)
+  end
+  return img_list, #result
+end
+
+-- don't leave files laying around the operating system
+
+local function cleanup(img_list)
+  dt.print_log("image list is " .. img_list)
+  files = du.split(img_list, " ")
+  for _,f in ipairs(files) do
+    f = string.gsub(f, '[\'\"]', "")
+    os.remove(f)
+  end
+end
+
+-- List files based on a search pattern.  This is cross platform compatible
+-- but the windows version is recursive in order to get ls type listings.
+-- Normally this shouldn't be a problem, but if you use this code just beware.
+-- If you want to do it non recursively, then remove the /s argument from dir
+-- and grab the path component from the search string and prepend it to the files
+-- found.
+
+local function list_files(search_string)
+  local ls = "ls "
+  local files = {}
+  local dir_path = nil
+  local count = 1
+
+  if dt.configuration.running_os == "windows" then
+    ls = "dir /b/s "
+    search_string = string.gsub(search_string, "/", "\\\\")
+  end
+
+  local f = io.popen(ls .. search_string)
+  if f then
+    local found_file = f:read()
+    while found_file do 
+      files[count] = found_file
+      count = count + 1
+      found_file = f:read()
+    end
+    f:close()
+  end
+  return files
+end
+
+-- create a filename from a multi image set.  The image list is sorted, then
+-- combined with first and last if more than 3 images or a - separated list
+-- if three images or less.
+
+local function make_output_filename(image_table)
+  local images = {}
+  local cnt = 1
+  local max_distinct_names = 3
+  local name_separator = "-"
+  local outputFileName = nil
+  local result = {}
+
+  for img,expimg in pairs(image_table) do
+    table.insert(result, expimg)
+  end
+  table.sort(result)
+  for _,img in pairs(result) do
+    images[cnt] = df.get_basename(img)
+    cnt = cnt + 1
+  end
+
+  cnt = cnt - 1
+
+  if cnt > 1 then
+    if cnt > max_distinct_names then
+      -- take the first and last
+      outputFileName = images[1] .. name_separator .. images[cnt]
+    else
+      -- join them
+      outputFileName = du.join(images, name_separator)
+    end
+  else
+    -- return the single name
+    outputFileName = images[cnt]
+  end
+
+  return outputFileName
+end
+
+-- get the path where the collection is stored
+
+local function extract_collection_path(image_table)
+  local collection_path = nil
+  for i,_ in pairs(image_table) do
+    collection_path = i.path
+    break
+  end
+  return collection_path
+end
+
+-- copy an images database attributes to another image.  This only
+-- copies what the database knows, not the actual exif data in the 
+-- image itself.
+
+local function copy_image_attributes(from, to, ...)
+  local args = {...}
+  if #args == 0 then
+    args[1] = "all"
+  end
+  if args[1] == "all" then
+    args[1] = "rating"
+    args[2] = "colors"
+    args[3] = "exif"
+    args[4] = "meta"
+    args[5] = "GPS"
+  end
+  for _,arg in ipairs(args) do
+    if arg == "rating" then
+      to.rating = from.rating
+    elseif arg == "colors" then
+      to.red = from.red
+      to.blue = from.blue
+      to.green = from.green
+      to.yellow = from.yellow
+      to.purple = from.purple
+    elseif arg == "exif" then
+      to.exif_maker = from.exif_maker
+      to.exif_model = from.exif_model
+      to.exif_lens = from.exif_lens
+      to.exif_aperture = from.exif_aperture
+      to.exif_exposure = from.exif_exposure
+      to.exif_focal_length = from.exif_focal_length
+      to.exif_iso = from.exif_iso
+      to.exif_datetime_taken = from.exif_datetime_taken
+      to.exif_focus_distance = from.exif_focus_distance
+      to.exif_crop = from.exif_crop
+    elseif arg == "GPS" then
+      to.elevation = from.elevation
+      to.longitude = from.longitude
+      to.latitude = from.latitude
+    elseif arg == "meta" then
+      to.publisher = from.publisher
+      to.title = from.title
+      to.creator = from.creator
+      to.rights = from.rights
+      to.description = from.description
+    else
+      dt.print_error(_("Unrecognized option to copy_image_attributes: " .. arg))
+    end
+  end
+end
+
+local function stop_job()
+  job.valid = false
+end
+
+-- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - 
+--  main program
+-- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - 
+
+local function image_stack(storage, image_table, extra_data)
+
+  local will_align = chkbtn_will_align.value
+  local img_list, image_count = extract_image_list(image_table)
+  local tmp_dir = dt.configuration.tmp_dir .. PS
+  local stack_function = cmbx_stack_function.value
+  local output_format = cmbx_output_format.value
+  local tag_source = chkbtn_tag_source_file.value
+  local tasks = 3
+
+  if will_align then
+    tasks = tasks + 1
+  end
+
+  if tag_source then
+    tasks = tasks + 1
+  end
+
+  local percent_step = 1 / tasks
+
+  -- update preferences
+  dt.preferences.write("image_stack", "align_images", "bool", will_align)
+  dt.preferences.write("image_stack", "stack_function", "integer", cmbx_stack_function.selected)
+  dt.preferences.write("image_stack", "output_format", "integer", cmbx_output_format.selected)
+
+  if image_count < 2 then
+    dt.print(_("ERROR: at least 2 images required for image stacking, exiting..."))
+    dt.print_error(_(image_count .. " image(s) selected, at least 2 required"))
+    return
+  end
+
+  job = dt.gui.create_job("image stack", true, stop_job)
+  job.percent = job.percent + percent_step
+
+  -- align images if requested
+  if will_align then
+    local align_image_stack_executable = df.check_if_bin_exists("align_image_stack")
+    if align_image_stack_executable then
+      local align_args = get_align_image_stack_arguments()
+      local align_image_stack_command = align_image_stack_executable .. align_args .. " " .. img_list
+      dt.print_log(align_image_stack_command)
+      dt.print(_("aligning images..."))
+      local result = dtsys.external_command(align_image_stack_command)
+      if result == 0 then
+        dt.print(_("images aligned"))
+        local files = list_files(tmp_dir .. "aligned_*")
+        cleanup(img_list)
+        img_list = extract_image_list(files)
+        job.percent = job.percent + percent_step
+      else
+        dt.print(_("ERROR: image alignment failed"))
+        dt.print_error(_("image alignment failed"))
+        cleanup(img_list)
+        return
+      end
+    else
+      dt.print(_("ERROR: align_image_stack not found"))
+      dt.print_error(_("align_image_stack not found"))
+      cleanup(img_list)
+      return
+    end
+  end
+
+  -- stack images
+  local output_filename = tmp_dir .. make_output_filename(image_table) .. "." .. string.lower(output_format)
+  local convert_arguments = img_list .. " -evaluate-sequence " .. stack_function .. " " .. df.sanitize_filename(output_filename)
+  local convert_executable = df.check_if_bin_exists("convert")
+  -- tif images exported from darktable have private tiff tags embedded in them and convert complains about them
+  local ignore_tif_tags = " -quiet -define tiff:ignore-tags=40965,42032,42033,42034,42036,18246,18249,36867,34864,34866 "
+  if convert_executable then
+    local convert_command = convert_executable .. ignore_tif_tags .. convert_arguments
+    dt.print_log(_("convert command is " .. convert_command))
+    dt.print(_("processing image stack"))
+    local result = dtsys.external_command(convert_command)
+    if result == 0 then
+      dt.print(_("image stack processed"))
+      cleanup(img_list)
+      job.percent = job.percent + percent_step
+
+      -- import image
+      dt.print(_("importing result"))
+      local film_roll_path = extract_collection_path(image_table)
+      local import_filename = df.create_unique_filename(film_roll_path .. PS .. df.get_filename(output_filename))
+      df.file_move(output_filename, import_filename)
+      imported_image = dt.database.import(import_filename)
+      local created_tag = dt.tags.create(_("Created with|image_stack"))
+      dt.tags.attach(created_tag, imported_image)
+      -- all the images are the same except for time, so just copy the  attributes
+      -- from the first
+      for img,_ in pairs(image_table) do
+        copy_image_attributes(img, imported_image)
+        break
+      end
+      job.percent = job.percent + percent_step
+
+      -- tag images if requested
+
+      if tag_source then
+        dt.print(_("tagging source images"))
+        local source_tag = dt.tags.create(_("Source file|" .. imported_image.filename))
+        for img, _ in pairs(image_table) do 
+          dt.tags.attach(source_tag, img)
+        end
+        job.percent = job.percent + percent_step
+      end
+    else
+      dt.print(_("ERROR: image stack processing failed"))
+      cleanup(img_list)
+    end
+  else
+    dt.print(_("ERROR: convert executable not found"))
+    dt.print_error(_("convert executable not found"))
+    cleanup(img_list)
+  end
+  job.valid = false
+end
+
+-- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - 
+--  darktable integration
+-- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - 
+
+dt.preferences.register("align_image_stack", "align_use_gpu", -- name
+  "bool",                                                     -- type
+  _('align image stack: use GPU for remaping'),               -- label
+  _('set the GPU remapping for image align'),                 -- tooltip
+  false)
+
+dt.register_storage("module_image_stack", _("image stack"), show_status, image_stack, nil, nil, image_stack_widget)
+

contrib/kml_export.lua

@@ -1,412 +1,373 @@
 --[[
-    This file is part of darktable,
-    Copyright 2016 by Tobias Jakobs.
-    Copyright 2016 by Erik Augustin.
+  This file is part of darktable,
+  Copyright 2018 by Tobias Jakobs.
+  Copyright 2018 by Erik Augustin.
 
-    This program is free software: you can redistribute it and/or modify
-    it under the terms of the GNU General Public License as published by
-    the Free Software Foundation; either version 3 of the License, or
-    (at your option) any later version.
+  This program is free software: you can redistribute it and/or modify
+  it under the terms of the GNU General Public License as published by
+  the Free Software Foundation; either version 3 of the License, or
+  (at your option) any later version.
 
-    This program is distributed in the hope that it will be useful,
-    but WITHOUT ANY WARRANTY; without even the implied warranty of
-    MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
-    GNU General Public License for more details.
+  This program is distributed in the hope that it will be useful,
+  but WITHOUT ANY WARRANTY; without even the implied warranty of
+  MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+  GNU General Public License for more details.
 
-    You should have received a copy of the GNU General Public License
-    along with this program.  If not, see <http://www.gnu.org/licenses/>.
+  You should have received a copy of the GNU General Public License
+  along with this program.  If not, see <http://www.gnu.org/licenses/>.
 ]]
 --[[
 darktable KML export script
 
 ADDITIONAL SOFTWARE NEEDED FOR THIS SCRIPT
-* mkdir
-* zip (only if you create KMZ files)
-* convert (ImageMagick)
-* xdg-open
-* xdg-user-dir
+* zip (at the moment Linux only and only if you create KMZ files)
+* magick (ImageMagick)
+* xdg-user-dir (Linux)
 
 WARNING
 This script is only tested with Linux
 
 USAGE
-* require script "official/yield" from your main Lua file in the first line
 * require this script from your main Lua file
 * when choosing file format, pick JPEG or PNG as Google Earth doesn't support other formats
 
 ]]
 
 local dt = require "darktable"
-require "official/yield"
+local du = require "lib/dtutils"
+local df = require "lib/dtutils.file"
+local ds = require "lib/dtutils.string"
+local dsys = require "lib/dtutils.system"
 
 local gettext = dt.gettext
-dt.configuration.check_version(...,{3,0,0})
+
+local PS = dt.configuration.running_os == "windows" and "\\" or "/"
+
+du.check_min_api_version("5.0.0", kml_export) 
 
 -- Tell gettext where to find the .mo file translating messages for a particular domain
-gettext.bindtextdomain("kml_export",dt.configuration.config_dir.."/lua/")
+gettext.bindtextdomain("kml_export",dt.configuration.config_dir.."/lua/locale/")
 
 local function _(msgid)
-    return gettext.dgettext("kml_export", msgid)
-end
-
--- Sort a table
-local function spairs(_table, order) -- Code copied from http://stackoverflow.com/questions/15706270/sort-a-table-in-lua
-    -- collect the keys
-    local keys = {}
-    for _key in pairs(_table) do keys[#keys + 1] = _key end
-
-    -- if order function given, sort by it by passing the table and keys a, b,
-    -- otherwise just sort the keys 
-    if order then
-        table.sort(keys, function(a,b) return order(_table, a, b) end)
-    else
-        table.sort(keys)
-    end
-
-    -- return the iterator function
-    local i = 0
-    return function()
-        i = i + 1
-        if keys[i] then
-            return keys[i], _table[keys[i]]
-        end
-    end
+  return gettext.dgettext("kml_export", msgid)
 end
 
 local function show_status(storage, image, format, filename, number, total, high_quality, extra_data)
-    dt.print(string.format(_("Export Image %i/%i"), number, total))
+  dt.print(string.format(_("Export Image %i/%i"), number, total))
 end
 
-local function checkIfBinExists(bin)
-    local handle = io.popen("which "..bin)
-    local result = handle:read()
-    local ret
-    handle:close()
-    if (not result) then
-        dt.print_error(bin.." not found")
-        ret = false
-    end
-    ret = true
-    return ret
-end
- 
--- Strip accents from a string
--- Copied from https://forums.coronalabs.com/topic/43048-remove-special-characters-from-string/
-function string.stripAccents( str )
-  local tableAccents = {}
-    tableAccents["à"] = "a"
-    tableAccents["á"] = "a"
-    tableAccents["â"] = "a"
-    tableAccents["ã"] = "a"
-    tableAccents["ä"] = "a"
-    tableAccents["ç"] = "c"
-    tableAccents["è"] = "e"
-    tableAccents["é"] = "e"
-    tableAccents["ê"] = "e"
-    tableAccents["ë"] = "e"
-    tableAccents["ì"] = "i"
-    tableAccents["í"] = "i"
-    tableAccents["î"] = "i"
-    tableAccents["ï"] = "i"
-    tableAccents["ñ"] = "n"
-    tableAccents["ò"] = "o"
-    tableAccents["ó"] = "o"
-    tableAccents["ô"] = "o"
-    tableAccents["õ"] = "o"
-    tableAccents["ö"] = "o"
-    tableAccents["ù"] = "u"
-    tableAccents["ú"] = "u"
-    tableAccents["û"] = "u"
-    tableAccents["ü"] = "u"
-    tableAccents["ý"] = "y"
-    tableAccents["ÿ"] = "y"
-    tableAccents["À"] = "A"
-    tableAccents["Á"] = "A"
-    tableAccents["Â"] = "A"
-    tableAccents["Ã"] = "A"
-    tableAccents["Ä"] = "A"
-    tableAccents["Ç"] = "C"
-    tableAccents["È"] = "E"
-    tableAccents["É"] = "E"
-    tableAccents["Ê"] = "E"
-    tableAccents["Ë"] = "E"
-    tableAccents["Ì"] = "I"
-    tableAccents["Í"] = "I"
-    tableAccents["Î"] = "I"
-    tableAccents["Ï"] = "I"
-    tableAccents["Ñ"] = "N"
-    tableAccents["Ò"] = "O"
-    tableAccents["Ó"] = "O"
-    tableAccents["Ô"] = "O"
-    tableAccents["Õ"] = "O"
-    tableAccents["Ö"] = "O"
-    tableAccents["Ù"] = "U"
-    tableAccents["Ú"] = "U"
-    tableAccents["Û"] = "U"
-    tableAccents["Ü"] = "U"
-    tableAccents["Ý"] = "Y"
-        
-  local normalizedString = ""
-
-  for strChar in string.gmatch(str, "([%z\1-\127\194-\244][\128-\191]*)") do
-    if tableAccents[strChar] ~= nil then
-      normalizedString = normalizedString..tableAccents[strChar]
-    else
-      normalizedString = normalizedString..strChar
+-- Add duplicate index to filename
+-- image.filename does not have index, exported_image has index
+function addDuplicateIndex( index, filename )
+  if index > 0 then
+    filename = filename.."_"
+    if index < 10 then
+      filename = filename.."0"
     end
+    filename = filename..index
   end
 
-  return normalizedString
- 
-end
-
--- Escape XML characters
--- Keep &amp; first, otherwise it will double escape other characters
--- https://stackoverflow.com/questions/1091945/what-characters-do-i-need-to-escape-in-xml-documents
-function string.escapeXmlCharacters( str )
-
-    str = string.gsub(str,"&", "&amp;")
-    str = string.gsub(str,"\"", "&quot;")
-    str = string.gsub(str,"'", "&apos;")
-    str = string.gsub(str,"<", "&lt;")
-    str = string.gsub(str,">", "&gt;")
-
-    return str
+  return filename 
 end
 
 local function create_kml_file(storage, image_table, extra_data)
 
-    if not checkIfBinExists("mkdir") then
-        return
-    end
-    if not checkIfBinExists("convert") then
-        return
-    end
-    if not checkIfBinExists("xdg-open") then
-        return
-    end
-    if not checkIfBinExists("xdg-user-dir") then
-        return
-    end
+  local magickPath 
+  if dt.configuration.running_os == "linux" then
+    magickPath = 'convert'
+  else
+    magickPath = dt.preferences.read("kml_export","magickPath","string")
+  end
 
-    dt.print_error("Will try to export KML file now")
+  if not df.check_if_bin_exists(magickPath) then
+    dt.print_error(_("magick not found"))
+    return
+  end
+  if dt.configuration.running_os == "linux" then
+    if not df.check_if_bin_exists("xdg-user-dir") then
+      dt.print_error(_("xdg-user-dir not found"))
+      return
+    end
+  end
+  dt.print_log("Will try to export KML file now")
 
-    local imageFoldername
-    if ( dt.preferences.read("kml_export","CreateKMZ","bool") == true ) then
-        if not checkIfBinExists("zip") then
-            return
-        end
+  local imageFoldername
+  if ( dt.preferences.read("kml_export","CreateKMZ","bool") == true 
+       and dt.configuration.running_os == "linux") then
+    if not df.check_if_bin_exists("zip") then
+      dt.print_error(_("zip not found"))
+      return
+    end
+    exportDirectory = dt.configuration.tmp_dir
+    imageFoldername = ""
+  else
+    exportDirectory = dt.preferences.read("kml_export","ExportDirectory","string")
+    -- Creates dir if not exsists
+    imageFoldername = "files"..PS
+    df.mkdir(df.sanitize_filename(exportDirectory..PS..imageFoldername))
+  end
+
+  -- Create the thumbnails
+  for image,exported_image in pairs(image_table) do
+    if ((image.longitude and image.latitude) and
+      (image.longitude ~= 0 and image.latitude ~= 90) -- Sometimes the north-pole but most likely just wrong data
+       ) then
+      local path, filename, filetype = string.match(exported_image, "(.-)([^\\/]-%.?([^%.\\/]*))$")
+      filename = string.upper(string.gsub(filename,"%.%w*", ""))
+
+      -- magick -size 92x92 filename.jpg -resize 92x92 +profile "*" thumbnail.jpg
+      -- In this example, '-size 120x120' gives a hint to the JPEG decoder that the image is going to be downscaled to
+      -- 120x120, allowing it to run faster by avoiding returning full-resolution images to  GraphicsMagick for the
+      -- subsequent resizing operation. The '-resize 120x120' specifies the desired dimensions of the output image. It
+      -- will be scaled so its largest dimension is 120 pixels. The '+profile "*"' removes any ICM, EXIF, IPTC, or other
+      -- profiles that might be present in the input and aren't needed in the thumbnail.
+
+      local convertToThumbCommand = ds.sanitize(magickPath) .. " -size 96x96 " .. exported_image .. " -resize 92x92 -mattecolor \"#FFFFFF\" -frame 2x2 +profile \"*\" " .. exportDirectory .. PS .. imageFoldername .. "thumb_" .. filename .. ".jpg"
+
+      if (exported_image ~= exportDirectory..PS..imageFoldername..filename.."."..filetype) then
+        df.file_copy(exported_image, exportDirectory..PS..imageFoldername..filename.."."..filetype)
+	    end
+      dsys.external_command(convertToThumbCommand)
 
-        exportDirectory = dt.configuration.tmp_dir
-        imageFoldername = ""
     else
-        exportDirectory = dt.preferences.read("kml_export","ExportDirectory","string")
-
-        -- Creates dir if not exsists
-        imageFoldername = "files/"
-        local mkdirCommand = "mkdir -p "..exportDirectory.."/"..imageFoldername
-        dt.control.execute(mkdirCommand) 
+      -- Remove exported image if it has no GPS data
+      os.remove(exported_image)
     end
 
+    local pattern = "[/]?([^/]+)$"
+    filmName = string.match(image.film.path, pattern)
 
-    -- Create the thumbnails
-    for image,exported_image in pairs(image_table) do
-        if ((image.longitude and image.latitude) and 
-            (image.longitude ~= 0 and image.latitude ~= 90) -- Sometimes the north-pole but most likely just wrong data
-           ) then
-            local path, filename, filetype = string.match(exported_image, "(.-)([^\\/]-%.?([^%.\\/]*))$")
-            filename = string.upper(string.gsub(filename,"%.%w*", ""))
+    -- Strip accents from the filename, because GoogleEarth can't open them
+    -- https://github.com/darktable-org/lua-scripts/issues/54
+    filmName = ds.strip_accents(filmName)
 
-            -- convert -size 92x92 filename.jpg -resize 92x92 +profile "*" thumbnail.jpg
-            --	In this example, '-size 120x120' gives a hint to the JPEG decoder that the image is going to be downscaled to 
-            --	120x120, allowing it to run faster by avoiding returning full-resolution images to  GraphicsMagick for the 
-            --	subsequent resizing operation. The '-resize 120x120' specifies the desired dimensions of the output image. It 
-            --	will be scaled so its largest dimension is 120 pixels. The '+profile "*"' removes any ICM, EXIF, IPTC, or other
-            --	profiles that might be present in the input and aren't needed in the thumbnail.
+    -- Remove chars we don't like to have in filenames
+    filmName = string.gsub(filmName, [[\]], "") 
+    filmName = string.gsub(filmName, [[/]], "")
+    filmName = string.gsub(filmName, [[:]], "") 
+    filmName = string.gsub(filmName, [["]], "")
+    filmName = string.gsub(filmName, "<", "") 
+    filmName = string.gsub(filmName, ">", "") 
+    filmName = string.gsub(filmName, "|", "")
+    filmName = string.gsub(filmName, "*", "")
+    filmName = string.gsub(filmName, "?", "")
+    filmName = string.gsub(filmName,'[.]', "") -- At least Windwows has problems with the "." and the start command
+  end
 
-            local convertToThumbCommand = "convert -size 96x96 "..exported_image.." -resize 92x92 -mattecolor \"#FFFFFF\" -frame 2x2 +profile \"*\" "..exportDirectory.."/"..imageFoldername.."thumb_"..filename..".jpg"
-            dt.control.execute(convertToThumbCommand)
-        else
-            -- Remove exported image if it has no GPS data
-            os.remove(exported_image)
-        end
+  exportKMLFilename = filmName..".kml"
+  exportKMZFilename = filmName..".kmz"
 
-        local pattern = "[/]?([^/]+)$"
-        filmName = string.match(image.film.path, pattern)
+  -- Create the KML file
+  local kml_file = "<?xml version=\"1.0\" encoding=\"UTF-8\"?>\n"
+  kml_file = kml_file.."<kml xmlns=\"http://www.opengis.net/kml/2.2\">\n"
+  kml_file = kml_file.."<Document>\n"
 
-        -- Strip accents from the filename, because GoogleEarth can't open them
-        -- https://github.com/darktable-org/lua-scripts/issues/54
-        filmName = string.stripAccents(filmName)
-    end
+  --image_table = dt.gui.selection();
+  kml_file = kml_file.."<name>"..filmName.."</name>\n"
+  kml_file = kml_file.."    <description>Exported from darktable</description>\n"
 
-    exportKMLFilename = filmName..".kml"
-    exportKMZFilename = filmName..".kmz"
-
-    -- Create the KML file
-    local kml_file = "<?xml version=\"1.0\" encoding=\"UTF-8\"?>\n"
-    kml_file = kml_file.."<kml xmlns=\"http://www.opengis.net/kml/2.2\">\n"
-    kml_file = kml_file.."<Document>\n"
-
-    --image_table = dt.gui.selection();
-    kml_file = kml_file.."<name>"..filmName.."</name>\n" 
-    kml_file = kml_file.."    <description>Exported from darktable</description>\n"
-    
-    for image,exported_image in pairs(image_table) do
+  for image,exported_image in pairs(image_table) do
     -- Extract filename, e.g DSC9784.ARW -> DSC9784
     filename = string.upper(string.gsub(image.filename,"%.%w*", ""))
+    -- Handle duplicates
+    filename = addDuplicateIndex( image.duplicate_index, filename )
     -- Extract extension from exported image (user can choose JPG or PNG), e.g DSC9784.JPG -> .JPG
     extension = string.match(exported_image,"%.%w*$")
 
-	if ((image.longitude and image.latitude) and 
-            (image.longitude ~= 0 and image.latitude ~= 90) -- Sometimes the north-pole but most likely just wrong data
-           ) then
-            kml_file = kml_file.."  <Placemark>\n"
-	    
-            local image_title, image_description
-            if (image.title and image.title ~= "") then
-                image_title = string.escapeXmlCharacters(image.title)
-            else
-                image_title = image.filename
-            end
-            -- Characters should not be escaped in CDATA, but we are using HTML fragment, so we must escape them
-            image_description = string.escapeXmlCharacters(image.description)
-
-            kml_file = kml_file.."    <name>"..image_title.."</name>\n"            
-            kml_file = kml_file.."    <description>"..image_description.."</description>\n"
-            kml_file = kml_file.."    <Style>\n"
-            kml_file = kml_file.."      <IconStyle>\n"
-            kml_file = kml_file.."        <Icon>\n"
-            kml_file = kml_file.."          <href>"..imageFoldername.."thumb_"..filename..".jpg</href>\n" 
-            kml_file = kml_file.."        </Icon>\n"
-            kml_file = kml_file.."      </IconStyle>\n"
-            kml_file = kml_file.."      <BalloonStyle>\n"
-            kml_file = kml_file.."        <text><![CDATA[<p><b>"..image_title.."</b></p><img src=\""..imageFoldername..filename..extension.."\"><p>"..image_description.."</p>]]></text>\n"
-            kml_file = kml_file.."        <textColor>ff000000</textColor>\n"
-            kml_file = kml_file.."        <displayMode>default</displayMode>\n"
-            kml_file = kml_file.."      </BalloonStyle>\n"
-            kml_file = kml_file.."    </Style>\n"
-
-            kml_file = kml_file.."    <Point>\n"
-            kml_file = kml_file.."      <extrude>1</extrude>\n"
-            kml_file = kml_file.."      <coordinates>"..string.gsub(tostring(image.longitude),",", ".")..","..string.gsub(tostring(image.latitude),",", ".")..",0</coordinates>\n"
-            kml_file = kml_file.."      <TimeStamp>\n"
-            kml_file = kml_file.."        <when>"..string.gsub(image.exif_datetime_taken," ", "T").."Z".."</when>\n"
-            kml_file = kml_file.."      </TimeStamp>\n"
-            kml_file = kml_file.."    </Point>\n"
-        
-            kml_file = kml_file.."  </Placemark>\n"
-        end
-    end
-
--- Connects all images with an path
-    if ( dt.preferences.read("kml_export","CreatePath","bool") == true ) then
+    if ((image.longitude and image.latitude) and
+      (image.longitude ~= 0 and image.latitude ~= 90) -- Sometimes the north-pole but most likely just wrong data
+       ) then
       kml_file = kml_file.."  <Placemark>\n"
-      kml_file = kml_file.."    <name>Path</name>\n"  -- ToDo: I think a better name would be nice
-      --kml_file = kml_file.."    <description></description>\n"
 
-      kml_file = kml_file.."   <Style>\n" 
-      kml_file = kml_file.."     <LineStyle>\n"  
-      kml_file = kml_file.."       <color>ff0000ff</color>\n"
-      kml_file = kml_file.."       <width>5</width>\n"
-      kml_file = kml_file.."     </LineStyle>\n" 
-      kml_file = kml_file.."   </Style>\n"
-
-      kml_file = kml_file.."    <LineString>\n"
-      kml_file = kml_file.."      <coordinates>\n"
-      
-      for image,exported_image in spairs(image_table, function(t,a,b) return t[b] < t[a] end) do
-	  if ((image.longitude and image.latitude) and 
-              (image.longitude ~= 0 and image.latitude ~= 90) -- Sometimes the north-pole but most likely just wrong data
-             ) then
-              local altitude = 0;
-              if (image.elevation) then
-                altitude = image.elevation;
-              end
-              kml_file = kml_file.."        "..string.gsub(tostring(image.longitude),",", ".")..","..string.gsub(tostring(image.latitude),",", ".")..",altitude\n"
-          end
+      local image_title, image_description
+      if (image.title and image.title ~= "") then
+        image_title = ds.escape_xml_characters(image.title)
+      else
+        image_title = filename..extension
       end
-      kml_file = kml_file.."      </coordinates>\n"
-      kml_file = kml_file.."    </LineString>\n"
+      -- Characters should not be escaped in CDATA, but we are using HTML fragment, so we must escape them
+      image_description = ds.escape_xml_characters(image.description)
+
+      kml_file = kml_file.."    <name>"..image_title.."</name>\n"
+      kml_file = kml_file.."    <description>"..image_description.."</description>\n"
+      kml_file = kml_file.."    <Style>\n"
+      kml_file = kml_file.."      <IconStyle>\n"
+      kml_file = kml_file.."        <Icon>\n"
+      kml_file = kml_file.."          <href>"..imageFoldername.."thumb_"..filename..".jpg</href>\n"
+      kml_file = kml_file.."        </Icon>\n"
+      kml_file = kml_file.."      </IconStyle>\n"
+      kml_file = kml_file.."      <BalloonStyle>\n"
+      kml_file = kml_file.."        <text><![CDATA[<p><b>"..image_title.."</b></p><img src=\""..imageFoldername..filename..extension.."\"><p>"..image_description.."</p>]]></text>\n"
+      kml_file = kml_file.."        <textColor>ff000000</textColor>\n"
+      kml_file = kml_file.."        <displayMode>default</displayMode>\n"
+      kml_file = kml_file.."      </BalloonStyle>\n"
+      kml_file = kml_file.."    </Style>\n"
+
+      kml_file = kml_file.."    <Point>\n"
+      kml_file = kml_file.."      <extrude>1</extrude>\n"
+      kml_file = kml_file.."      <coordinates>"..string.gsub(tostring(image.longitude),",", ".")..","..string.gsub(tostring(image.latitude),",", ".")..",0</coordinates>\n"
+      kml_file = kml_file.."      <TimeStamp>\n"
+      kml_file = kml_file.."        <when>"..string.gsub(image.exif_datetime_taken," ", "T").."Z".."</when>\n"
+      kml_file = kml_file.."      </TimeStamp>\n"
+      kml_file = kml_file.."    </Point>\n"
 
       kml_file = kml_file.."  </Placemark>\n"
     end
+  end
 
-    kml_file = kml_file.."</Document>\n"
-    kml_file = kml_file.."</kml>"
+  -- Connects all images with an path
+  if ( dt.preferences.read("kml_export","CreatePath","bool") == true ) then
+    kml_file = kml_file.."  <Placemark>\n"
+    kml_file = kml_file.."    <name>Path</name>\n"  -- ToDo: I think a better name would be nice
+    --kml_file = kml_file.."    <description></description>\n"
 
-    local file = io.open(exportDirectory.."/"..exportKMLFilename, "w")
-    file:write(kml_file)
-    file:close()
+    kml_file = kml_file.."   <Style>\n"
+    kml_file = kml_file.."     <LineStyle>\n"
+    kml_file = kml_file.."       <color>ff0000ff</color>\n"
+    kml_file = kml_file.."       <width>5</width>\n"
+    kml_file = kml_file.."     </LineStyle>\n"
+    kml_file = kml_file.."   </Style>\n"
 
-    dt.print("KML file created in "..exportDirectory)
+    kml_file = kml_file.."    <LineString>\n"
+    kml_file = kml_file.."      <coordinates>\n"
 
--- Compress the files to create a KMZ file
-    if ( dt.preferences.read("kml_export","CreateKMZ","bool") == true ) then
-       exportDirectory = dt.preferences.read("kml_export","ExportDirectory","string")
-
-        local createKMZCommand = "zip --test --move --junk-paths "
-        createKMZCommand = createKMZCommand .."\""..exportDirectory.."/"..exportKMZFilename.."\" "           -- KMZ filename
-        createKMZCommand = createKMZCommand .."\""..dt.configuration.tmp_dir.."/"..exportKMLFilename.."\" "  -- KML file
-
-        for image,exported_image in pairs(image_table) do
-          if ((image.longitude and image.latitude) and 
-              (image.longitude ~= 0 and image.latitude ~= 90) -- Sometimes the north-pole but most likely just wrong data
-             ) then
-            local filename = string.upper(string.gsub(image.filename,"%.%w*", ""))
-
-            createKMZCommand = createKMZCommand .."\""..dt.configuration.tmp_dir.."/"..imageFoldername.."thumb_"..filename..".jpg\" " -- thumbnails
-            createKMZCommand = createKMZCommand .."\""..exported_image.."\" "  -- images
-          end
+    for image,exported_image in du.spairs(image_table, function(t,a,b) return b.exif_datetime_taken > a.exif_datetime_taken end) do
+      if ((image.longitude and image.latitude) and
+        (image.longitude ~= 0 and image.latitude ~= 90) -- Sometimes the north-pole but most likely just wrong data
+         ) then
+        local altitude = 0;
+        if (image.elevation) then
+          altitude = image.elevation;
         end
 
-	dt.control.execute(createKMZCommand)
+        kml_file = kml_file.."        "..string.gsub(tostring(image.longitude),",", ".")..","..string.gsub(tostring(image.latitude),",", ".")..",altitude\n"
+      end
     end
 
--- Open the file with the standard programm    
-    if ( dt.preferences.read("kml_export","OpenKmlFile","bool") == true ) then
-	local kmlFileOpenCommand
+    kml_file = kml_file.."      </coordinates>\n"
+    kml_file = kml_file.."    </LineString>\n"
 
-        if ( dt.preferences.read("kml_export","CreateKMZ","bool") == true ) then
-            kmlFileOpenCommand = "xdg-open "..exportDirectory.."/\""..exportKMZFilename.."\""
-        else
-            kmlFileOpenCommand = "xdg-open "..exportDirectory.."/\""..exportKMLFilename.."\""
-	end
-        dt.control.execute(kmlFileOpenCommand) 
+    kml_file = kml_file.."  </Placemark>\n"
+  end
+
+  kml_file = kml_file.."</Document>\n"
+  kml_file = kml_file.."</kml>"
+
+  local file = io.open(exportDirectory..PS..exportKMLFilename, "w")
+
+  file:write(kml_file)
+  file:close()
+
+  dt.print("KML file created in "..exportDirectory)
+
+  -- Compress the files to create a KMZ file
+  if ( dt.preferences.read("kml_export","CreateKMZ","bool") == true 
+       and dt.configuration.running_os == "linux") then
+    exportDirectory = dt.preferences.read("kml_export","ExportDirectory","string")
+
+    local createKMZCommand = "zip --test --move --junk-paths "
+    createKMZCommand = createKMZCommand .."\""..exportDirectory..PS..exportKMZFilename.."\" "           -- KMZ filename
+    createKMZCommand = createKMZCommand .."\""..dt.configuration.tmp_dir..PS..exportKMLFilename.."\" "  -- KML file
+
+    for image,exported_image in pairs(image_table) do
+      if ((image.longitude and image.latitude) and
+        (image.longitude ~= 0 and image.latitude ~= 90) -- Sometimes the north-pole but most likely just wrong data
+         ) then
+        local filename = string.upper(string.gsub(image.filename,"%.%w*", ""))
+        -- Handle duplicates
+        filename = addDuplicateIndex( image.duplicate_index, filename )
+
+        createKMZCommand = createKMZCommand .."\""..dt.configuration.tmp_dir..PS..imageFoldername.."thumb_"..filename..".jpg\" " -- thumbnails
+        createKMZCommand = createKMZCommand .."\""..exported_image.."\" "  -- images
+      end
     end
 
+     dt.control.execute(createKMZCommand)
+  end
+
+  -- Open the file with the standard programm
+  if ( dt.preferences.read("kml_export","OpenKmlFile","bool") == true ) then
+    local path
+
+    if ( dt.preferences.read("kml_export","CreateKMZ","bool") == true
+	     and dt.configuration.running_os == "linux") then
+      path = exportDirectory..PS..exportKMZFilename
+    else
+      path = exportDirectory..PS..exportKMLFilename
+    end
+
+    dsys.launch_default_app(df.sanitize_filename(path))
+  end
 
 end
 
 -- Preferences
-dt.preferences.register("kml_export",
-	"OpenKmlFile",
-	"bool",
-	_("KML export: Open KML/KMZ file after export"),
-	_("Opens the KML file after the export with the standard program for KML files"),
-	false )
+if dt.configuration.running_os == "windows" then
+  dt.preferences.register("kml_export",
+    "OpenKmlFile",
+    "bool",
+    _("KML export: Open KML file after export"),
+    _("Opens the KML file after the export with the standard program for KML files"),
+    false )
+else
+  dt.preferences.register("kml_export",
+    "OpenKmlFile",
+    "bool",
+    _("KML export: Open KML/KMZ file after export"),
+    _("Opens the KML file after the export with the standard program for KML files"),
+    false )
+end
+
+local defaultDir = ''
+if dt.configuration.running_os == "windows" then
+  defaultDir = os.getenv("USERPROFILE")
+elseif dt.configuration.running_os == "macos" then
+  defaultDir =  os.getenv("home")
+else
+  local handle = io.popen("xdg-user-dir DESKTOP")
+  defaultDir = handle:read()
+  handle:close()
+end
 
-local handle = io.popen("xdg-user-dir DESKTOP")
-local result = handle:read()
-handle:close()
-dt.preferences.register("kml_export",
-	"ExportDirectory",
-	"directory",
-	_("KML export: Export directory"),
-	_("A directory that will be used to export the KML/KMZ files"),
-	result )
 
 dt.preferences.register("kml_export",
-	"CreatePath",
-	"bool",
-	_("KML export: Connect images with path"),
-	_("connect all images with a path"),
-	false )
+  "ExportDirectory",
+  "directory",
+  _("KML export: Export directory"),
+  _("A directory that will be used to export the KML/KMZ files"),
+  defaultDir )
+
+if dt.configuration.running_os ~= "linux" then  
+  dt.preferences.register("kml_export", 
+    "magickPath",	-- name
+	"file",	-- type
+	_("KML export: ImageMagick binary Location"),	-- label
+	_("Install location of magick[.exe]. Requires restart to take effect."),	-- tooltip
+	"magick")	-- default
+end  
+  
 dt.preferences.register("kml_export",
-	"CreateKMZ",
-	"bool",
-	_("KML export: Create KMZ file"),
-	_("Compress all imeges to one KMZ file"),
-	true )	
+  "CreatePath",
+  "bool",
+  _("KML export: Connect images with path"),
+  _("connect all images with a path"),
+  false )
+
+if dt.configuration.running_os == "linux" then  
+  dt.preferences.register("kml_export",
+    "CreateKMZ",
+    "bool",
+    _("KML export: Create KMZ file"),
+    _("Compress all imeges to one KMZ file"),
+    true )
+end
 
 -- Register
-dt.register_storage("kml_export", _("KML/KMZ Export"), nil, create_kml_file)
+if dt.configuration.running_os == "windows" then
+  dt.register_storage("kml_export", _("KML Export"), nil, create_kml_file)
+else
+  dt.register_storage("kml_export", _("KML/KMZ Export"), nil, create_kml_file)
+end
+
+-- vim: shiftwidth=2 expandtab tabstop=2 cindent syntax=lua
+-- kate: hl Lua;

contrib/passport_guide.lua

@@ -0,0 +1,95 @@
+--[[
+  Passport cropping guide for darktable
+
+  copyright (c) 2017  Kåre Hampf
+  
+  darktable is free software: you can redistribute it and/or modify
+  it under the terms of the GNU General Public License as published by
+  the Free Software Foundation, either version 3 of the License, or
+  (at your option) any later version.
+
+  darktable is distributed in the hope that it will be useful,
+  but WITHOUT ANY WARRANTY; without even the implied warranty of
+  MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+  GNU General Public License for more details.
+
+  You should have received a copy of the GNU General Public License
+  along with darktable.  If not, see <http://www.gnu.org/licenses/>.
+]]
+
+--[[
+PASSPORT CROPPING GUIDE
+guides for cropping passport photos based on documents from the Finnish police
+(https://www.poliisi.fi/instancedata/prime_product_julkaisu/intermin/embeds/poliisiwwwstructure/38462_Passikuvaohje_EN.pdf) describing passport photo dimensions of 47x36 mm and 500x653 px for digital biometric data stored in passports. They use ISO 19794-5 standard based on ICAO 9303 regulations which should also be compliant for all of Europe.
+
+INSTALLATION
+* copy this file in $CONFIGDIR/lua/ where CONFIGDIR is your darktable configuration directory
+* add the following line in the file $CONFIGDIR/luarc
+  require "passport_guide"
+* (optional) add the line:
+  "plugins/darkroom/clipping/extra_aspect_ratios/passport 36x47mm=47:36"
+  to $CONFIGDIR/darktablerc
+
+USAGE
+* when using the cropping tool, select "passport" as guide and if you added the line in yout rc
+  select "passport 36x47mm" as aspect
+]]
+
+local dt = require "darktable"
+local du = require "lib/dtutils"
+local gettext = dt.gettext
+
+du.check_min_api_version("2.0.0", "passport_guide") 
+
+-- Tell gettext where to find the .mo file translating messages for a particular domain
+gettext.bindtextdomain("passport_guide",dt.configuration.config_dir.."/lua/locale/")
+
+local function _(msgid)
+  return gettext.dgettext("passport_guide", msgid)
+end
+
+dt.guides.register_guide("passport",
+-- draw
+function(cairo, x, y, width, height, zoom_scale)
+  local _width, _height
+
+  -- get the max 36x47 rectangle
+  local aspect_ratio = 47 / 36
+  if width * aspect_ratio > height then
+    _width = height / aspect_ratio
+    _height = height
+  else
+    _width = width
+    _height = width * aspect_ratio
+  end
+
+  cairo:save()
+
+  cairo:translate(x + (width - _width) / 2, y + (height - _height) / 2)
+  cairo:scale(_width / 36, _height / 47)
+
+  -- the outer rectangle
+  cairo:rectangle( 0, 0, 36, 47)
+
+  -- vertical bars
+  cairo:draw_line(16.5, 8, 16.5, 36)
+  cairo:draw_line(19.5, 8, 19.5, 36)
+
+  -- long horisontal bars
+  cairo:draw_line(6, 4, 30, 4)
+  cairo:draw_line(6, 40, 30, 40)
+
+  -- short horisontal bars
+  cairo:draw_line(9, 6, 27, 6)
+  cairo:draw_line(9, 38, 27, 38)
+
+  cairo:restore()
+end,
+-- gui
+function()
+  return dt.new_widget("label"){label = _("ISO 19794-5/ICAO 9309 passport"), halign = "start"}
+end
+)
+
+-- kate: tab-indents: off; indent-width 2; replace-tabs on; remove-trailing-space on; hl Lua;
+-- vim: shiftwidth=2 expandtab tabstop=2 cindent syntax=lua

contrib/pdf_slideshow.lua

@@ -0,0 +1,248 @@
+--[[
+  This file is part of darktable,
+  copyright (c) 2017 Pascal Obry
+
+  darktable is free software: you can redistribute it and/or modify
+  it under the terms of the GNU General Public License as published by
+  the Free Software Foundation, either version 3 of the License, or
+  (at your option) any later version.
+
+  darktable is distributed in the hope that it will be useful,
+  but WITHOUT ANY WARRANTY; without even the implied warranty of
+  MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+  GNU General Public License for more details.
+
+  You should have received a copy of the GNU General Public License
+  along with darktable.  If not, see <http://www.gnu.org/licenses/>.
+]]
+--[[
+SELECTION_TO_PDF
+Generates a PDF slideshow (via Latex) containing all selected images
+one per slide.
+
+ADDITIANAL SOFTWARE NEEDED FOR THIS SCRIPT
+* a PDF-Viewer
+* pdflatex (Latex)
+
+USAGE
+* require this file from your main lua config file:
+
+This plugin will add a new exporter that will allow you to generate a pdf slideshow.
+The interface will let you add:
+   - a global title for the slideshow (prefix in all slide label)
+   - a delay for the transition between each slide
+
+Each slide will contain a single picture with a label at the bottom with the
+format (all fields can be the empty string):
+
+   <global title> / <image creator> / <image title>
+
+]]
+local dt = require "darktable"
+local du = require "lib/dtutils"
+local df = require "lib/dtutils.file"
+require "official/yield"
+
+local gettext = dt.gettext
+
+-- Tell gettext where to find the .mo file translating messages for a particular domain
+gettext.bindtextdomain("pdf_slideshow",dt.configuration.config_dir.."/lua/locale/")
+
+local function _(msgid)
+    return gettext.dgettext("pdf_slideshow", msgid)
+end
+
+if not df.check_if_bin_exists("pdflatex") then
+   dt.print_error(_("pdflatex not found"))
+   return
+end
+
+du.check_min_api_version("4.0.0", "pdf_slideshow") 
+
+dt.preferences.register
+   ("pdf_slideshow","open with","string",
+    _("a pdf viewer"),
+    _("can be an absolute pathname or the tool may be in the PATH"),
+    "xdg-open")
+
+local title_widget = dt.new_widget("entry") {
+    placeholder=_("slideshow title")
+}
+local delay_widget = dt.new_widget("slider")
+{
+    label = _("transition delay (s)"),
+    soft_min = 1,     -- The soft minimum value for the slider, the slider can't go beyond this point
+    soft_max = 20,    -- The soft maximum value for the slider, the slider can't go beyond this point
+    hard_min = 1,     -- The hard minimum value for the slider, the user can't manually enter a value beyond this point
+    hard_max = 60,    -- The hard maximum value for the slider, the user can't manually enter a value beyond this point
+    value = 5,        -- The current value of the slider
+    step = 1,
+    digits = 0
+}
+local check_button_title = dt.new_widget("check_button")
+{
+    label = _('include image title'),
+    value = true,
+    tooltip = _('whether to include the image title (if defined) into the slide')
+}
+local check_button_author = dt.new_widget("check_button")
+{
+    label = _('include image author'),
+    value = true,
+    tooltip = _('whether to include the image author (if defined) into the slide')
+}
+local widget = dt.new_widget("box") {
+    orientation=horizontal,
+    dt.new_widget("label"){label = _("slideshow title")},
+    title_widget,
+    dt.new_widget("label"){label = ""},
+    delay_widget,
+    check_button_author,
+    check_button_title
+}
+
+local ending = [[
+\end{document}
+]]
+
+--  only jpeg files are supported by latex
+local function support_format(storage, format)
+  fmt = string.lower(format.name)
+  if string.match(fmt,"jpeg") ~= nil and string.match(fmt,"2000") == nil then
+    return true
+  else
+    return false
+  end
+end
+
+local filename =dt.configuration.tmp_dir.."/pdfout.tex"
+
+local function my_write(latexfile,arg)
+  local res,errmsg = latexfile:write(arg)
+  if not res then
+    error(errmsg)
+  end
+end
+
+local function img_filename(file,i)
+  local dir=string.gsub(file, "(.*)(%/.*%..*)", "%1")
+  local ext=string.gsub(file, "(.*)(%..*)", "%2")
+  local lfile = dir.."/slide"..i..ext
+  return lfile
+end
+
+local function slide(latexfile,i,image,file)
+  local title = ""
+  if title_widget.text ~= "" then
+    title = title_widget.text
+  end
+  if check_button_author.value == true and image.creator ~= "" then
+    if title ~= "" then
+       title = title.." / "
+    end
+    title = title..image.creator
+  end
+  if check_button_title.value == true and image.title ~= "" then
+    if title ~= "" then
+       title = title.." / "
+    end
+    title = title..image.title
+  end
+
+  --  make sure there is no non-printable characters in the title
+  title = string.gsub(title, "\n", "")
+  title = string.gsub(title, "\r", "")
+  title = string.gsub(title, "\t", "")
+
+  local lfile = img_filename(file,i)
+  os.rename(file, lfile)
+
+  my_write(latexfile,"\\begin{figure}\n")
+  my_write(latexfile,"\\begin{measuredfigure}\n")
+  my_write(latexfile,"\\includegraphics[height=0.92\\textheight]{"..lfile.."}\n")
+  my_write(latexfile,"\\end{measuredfigure}\n")
+  my_write(latexfile,"\\caption{{\\color{white}"..title.."}}\n")
+  my_write(latexfile,"\\end{figure}\n")
+end
+
+dt.register_storage("pdf_slideshow",_("pdf slideshow"),
+    nil,
+    function(storage,image_table)
+
+    local preamble = [[
+    \documentclass[a4paper,12pt,landscape]{beamer}
+    \usetheme{default}
+    \usepackage[utf8]{inputenc}
+    \usepackage{graphicx}
+    \usepackage{threeparttable}
+    \usepackage[space]{grffile} % needed to support filename with spaces
+    \pagestyle{empty}
+    \parindent0pt
+    \usepackage{geometry}
+    \usepackage{color}
+    \pagecolor{black!100}
+    \color{white}
+    \geometry{a4paper,landscape,left=2mm,right=2mm, top=2mm, bottom=2mm}
+    \mode<presentation>
+    \transduration{]]..string.gsub(delay_widget.value, ",", ".")..[[}
+    \setbeamertemplate{footline}{}
+    \setbeamertemplate{headline}{}
+    \setbeamertemplate{frametitle}{}
+    \setbeamertemplate{navigation symbols}{}
+    \setbeamercolor{background canvas}{fg=white,bg=black!100}
+    \setbeamercolor{normal text}{fg=white}
+    \hypersetup{pdfstartpage=1,pdfpagemode=FullScreen}
+    \setbeamertemplate{caption}{\insertcaption}
+    \begin{document}
+    ]]
+
+    local errmsg
+    local latexfile
+    latexfile,errmsg=io.open(filename,"w")
+    if not latexfile then
+        error(errmsg)
+      end
+      my_write(latexfile,preamble)
+      local i = 1
+      for img,file in pairs(image_table) do
+         slide(latexfile,i,img,file)
+         my_write(latexfile,"\n\\bigskip\n")
+         i = i+1
+      end
+      my_write(latexfile,ending)
+      latexfile:close()
+
+      -- convert to PDF
+      local dir=string.gsub(filename, "(.*/)(.*)", "%1")
+      local locfile=string.gsub(filename, "(.*/)(.*)", "%2")
+      local command = "pdflatex -halt-on-error -output-directory "..dir.." "..locfile
+      local result = dt.control.execute(command)
+      if result ~= 0 then
+        dt.print(_("problem running pdflatex")) -- this one is probably usefull to the user
+        error(_("problem running ")..command)
+      end
+
+      -- open the PDF
+      local pdffile=string.gsub(filename, ".tex", ".pdf")
+      command = dt.preferences.read("pdf_slideshow","open with","string")
+      command = command.." "..pdffile
+      local result = dt.control.execute(command)
+      if result ~= 0 then
+        dt.print(_("problem running pdf viewer")) -- this one is probably usefull to the user
+        error(_("problem running ")..command)
+      end
+
+      -- finally do some clean-up
+      local i = 1
+      for img,file in pairs(image_table) do
+         local lfile = img_filename(file,i)
+         os.remove(lfile)
+         i = i+1
+      end
+    end,
+    support_format,
+    nil,
+    widget)
+
+--
+-- vim: shiftwidth=2 expandtab tabstop=2 cindent syntax=lua

contrib/quicktag.lua

@@ -0,0 +1,252 @@
+--[[
+    This file is part of darktable,
+    Copyright (C) 2017 by Christian Kanzian
+
+    Thanks to
+    Copyright (C) 2016 Bill Ferguson
+    Copyright (C) 2017 Tobias Jakobs
+
+    This program is free software: you can redistribute it and/or modify
+    it under the terms of the GNU General Public License as published by
+    the Free Software Foundation; either version 3 of the License, or
+    (at your option) any later version.
+
+    This program is distributed in the hope that it will be useful,
+    but WITHOUT ANY WARRANTY; without even the implied warranty of
+    MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+    GNU General Public License for more details.
+
+    You should have received a copy of the GNU General Public License
+    along with this program.  If not, see <http://www.gnu.org/licenses/>.
+]]
+
+--[[
+QUICKTAG
+   For faster attaching your favorite tags, the script adds shortcuts and
+   the module "quicktag" in lighttable mode with a changeable number of buttons.
+   To each button a tag can be assigned. If the tags do not exist in your database,
+   they are added to the database once they get the first time attached to an image.
+
+   The number of buttons/shortcuts can be changed in the lua preferences.
+   Changes in the number require a restart of darktable.
+
+INSTALATION
+   * copy this file in $CONFIGDIR/lua/ where CONFIGDIR is your darktable configuration directory
+   * add the following line in the file $CONFIGDIR/luarc require "quicktag"
+
+USAGE
+   * set the number of quicktags between 2 and 10 in the preferences dialog and restart darktable
+   * if wanted set the shortcuts in the preferences dialog
+   * to add or change a quicktag, first select the old tag with the combobox "old quicktag",
+     enter a new tag in the "new quicktag" filed and press "set quicktag"
+   * use a shortcut or button to attach the tag to selected images
+]]
+
+
+local dt = require "darktable"
+local du = require "lib/dtutils"
+local debug = require "darktable.debug"
+
+
+local gettext = dt.gettext
+
+du.check_min_api_version("3.0.0", "quicktag")
+
+-- Tell gettext where to find the .mo file translating messages for a particular domain
+gettext.bindtextdomain("quicktag",dt.configuration.config_dir.."/lua/locale/")
+
+local function _(msgid)
+    return gettext.dgettext("quicktag", msgid)
+end
+
+-- maximum length of button labels
+dt.preferences.register("quickTag",
+                        "labellength",
+                        "integer",                    -- type
+                        _("max. length of button labels"),           -- label
+                        _("may range from 15 to 60 - needs a restart"),   -- tooltip
+                       25,                            -- default
+                       15,                            -- min
+                      60)
+
+
+
+local max_label_length = dt.preferences.read("quickTag", "labellength", "integer")
+
+-- register number of quicktags
+dt.preferences.register("quickTag",
+                        "quickTagnumber",
+                        "integer",                    -- type
+                        _("number of quicktag fields"),           -- label
+                        _("may range from 2 to 20 - needs a restart"),   -- tooltip
+                        5,                            -- default
+                        2,                            -- min
+                        20)
+
+local qnr = dt.preferences.read("quickTag", "quickTagnumber", "integer")
+
+
+
+-- get quicktags from the preferences
+local quicktag_table = {}
+
+
+local function read_pref_tags()
+  for j=1,qnr do
+    quicktag_table[j] = dt.preferences.read("quickTag", "quicktag"..j, "string")
+    end
+  end
+
+
+local quicktag_label = {}
+
+
+local function abbrevate_tags(t)
+  for i=1,qnr do
+    local taglength = string.len(t[i])
+    if taglength > max_label_length then
+      local max_2 = math.floor(max_label_length/2)
+      quicktag_label[i] = string.sub(t[i],0,max_2).."..."..string.sub(t[i],taglength - max_2)
+    else
+      quicktag_label[i] = t[i]
+    end
+    -- print(quicktag_label[i])
+  end
+end
+
+
+-- quicktag function to attach tags
+local function tagattach(tag,qtagnr)
+  if tag == "" then
+    dt.print(string.format(_("quicktag %i is empty, please set a tag"), qtagnr))
+    return true
+  end
+
+  local tagnr = dt.tags.find(tag)
+
+--create tag if it does not exist
+  if tagnr == nil then
+    dt.tags.create(tag)
+    tagnr = dt.tags.find(tag)
+  end
+
+  local sel_images = dt.gui.action_images
+
+  if next(sel_images) == nil then
+    dt.print(_("no images selected"))
+    return true
+  end
+
+  local counter = 0
+
+  for _,image in ipairs(sel_images) do
+     dt.tags.attach(tagnr,image)
+     counter = counter+1
+  end
+  dt.print(string.format(_("tag \"%s\" attached to %i image(s)"), tag, counter))
+end
+
+
+-- create quicktag module elements
+-- read tags from preferences for initialization
+read_pref_tags()
+abbrevate_tags(quicktag_table)
+
+
+local button = {}
+
+-- function to create buttons with tags as labels
+for j=1,qnr do
+      button[#button+1] = dt.new_widget("button") {
+         label = j..": "..quicktag_table[j],
+         clicked_callback = function() tagattach(tostring(quicktag_table[j]),j) end}
+    end
+
+
+local old_quicktag = dt.new_widget("combobox"){
+    label = _("old tag"),
+    tooltip = _("select the quicktag to replace")
+}
+
+local function update_quicktag_list()
+  -- abrevate long tags for the labels
+  abbrevate_tags(quicktag_table)
+  for j=1,qnr do
+      button[j].label = j..": "..quicktag_label[j]
+      old_quicktag[j] = j..": "..quicktag_table[j]
+  end
+end
+
+update_quicktag_list()
+
+local new_quicktag = dt.new_widget("entry"){
+    text = "",
+    placeholder = _("new tag"),
+    is_password = true,
+    editable = true,
+    tooltip = _("enter your tag here")
+}
+
+local set_quicktag_button = dt.new_widget("button") {
+  label = _("set tag"),
+  clicked_callback = function()
+    local old_tag = quicktag_table[old_quicktag.selected]
+    if new_quicktag.text == "" then
+      dt.print(_("new quicktag is empty!"))
+    else
+      quicktag_table[old_quicktag.selected] = new_quicktag.text
+      dt.preferences.write("quickTag", "quicktag"..old_quicktag.selected, "string",  new_quicktag.text)
+      dt.print(string.format(_("quicktag \"%s\" replaced by \"%s\""), old_tag, new_quicktag.text))
+      update_quicktag_list()
+      new_quicktag.text = ""
+    end
+  end,
+}
+
+local new_qt_widget = dt.new_widget ("box") {
+    orientation = "horizontal",
+    dt.new_widget("label") { label = _("new tag") },
+    new_quicktag,
+    set_quicktag_button
+}
+
+
+-- back UI elements in a table
+-- thanks to wpferguson for the hint
+local widget_table = {}
+
+for i=1,qnr do
+  widget_table[#widget_table  + 1] =  button[i]
+end
+
+widget_table[#widget_table  + 1] = dt.new_widget("separator"){}
+widget_table[#widget_table  + 1] = old_quicktag
+widget_table[#widget_table  + 1] = new_qt_widget
+
+
+--create module
+dt.register_lib(
+  "quicktag",     -- Module name
+  "quicktag",     -- name
+  true,                -- expandable
+  false,               -- resetable
+  {[dt.gui.views.lighttable] = {"DT_UI_CONTAINER_PANEL_RIGHT_CENTER", 490}},
+
+  dt.new_widget("box"){
+    orientation = "vertical",
+    table.unpack(widget_table),
+
+  },
+  nil,-- view_enter
+  nil -- view_leave
+)
+
+-- create shortcuts
+for i=1,qnr do
+  dt.register_event("shortcut",
+		   function(event, shortcut) tagattach(tostring(quicktag_table[i])) end,
+		  string.format(_("quicktag %i"),i))
+end
+
+-- vim: shiftwidth=2 expandtab tabstop=2 cindent syntax=lua
+-- kate: tab-indents: off; indent-width 2; replace-tabs on; remove-trailing-space on;

contrib/rate_group.lua

@@ -39,6 +39,10 @@
 ]]
 
 local dt = require "darktable"
+local du = require "lib/dtutils"
+
+-- added version check
+du.check_min_api_version("3.0.0", "rate_group") 
 
 local function apply_rating(rating)
   local images = dt.gui.action_images

contrib/rename-tags.lua

@@ -0,0 +1,125 @@
+--[[
+  Copyright (c) 2016 Sebastian Witt <se.witt@gmx.net>
+  Copyright (c) 2018 Bill Ferguson <wpferguson@gmail.com>
+]]
+--[[
+Rename tags
+
+AUTHOR
+Sebastian Witt (se.witt@gmx.net)
+
+INSTALLATION
+* copy this file in $CONFIGDIR/lua/ where CONFIGDIR
+is your darktable configuration directory
+* add the following line in the file $CONFIGDIR/luarc
+  require "rename-tags"
+
+USAGE
+* In lighttable there is a new entry: 'rename tag'
+* Enter old tag (this one gets deleted!)
+* Enter new tag name
+
+LICENSE
+GPLv2
+
+Changes
+* 20180912 - Did an RTFM and read a bug (12277, thanks Christian G) that showed 
+  a way to get the images containing the old tag one by one instead of searching 
+  the entire database.  Changed rename_tags() function to use this method.
+]]
+
+local darktable = require "darktable"
+local du = require "lib/dtutils"
+local debug = require "darktable.debug"
+
+-- check API version
+du.check_min_api_version("3.0.0", "rename-tags") 
+
+-- GUI entries
+local old_tag = darktable.new_widget("entry") { tooltip = "Enter old tag" }
+local new_tag = darktable.new_widget("entry") { tooltip = "Enter new tag" }
+
+local function rename_reset()
+    old_tag.text = ''
+    new_tag.text = ''
+end
+
+-- This function does the renaming
+local function rename_tags()
+  -- If entries are empty, return
+  if old_tag.text == '' then
+    darktable.print ("Old tag can't be empty")
+    return
+  end
+  if new_tag.text == '' then
+    darktable.print ("New tag can't be empty")
+    return
+  end
+  
+  local count = 0
+
+  -- Check if old tag exists
+  local ot = darktable.tags.find (old_tag.text)
+  
+  if not ot then
+    darktable.print ("Old tag does not exist")
+    return
+  end
+
+  -- Show job
+  local job = darktable.gui.create_job ("Renaming tag", true)
+  
+  old_tag.editable = false
+  new_tag.editable = false
+
+  -- Create if it does not exists
+  local nt = darktable.tags.create (new_tag.text)
+
+  -- Get number of images for old tag
+  local dbcount = #ot
+
+  -- loop through the images containing the old tag, and attach the new tag
+  for i=1, #ot do
+    -- Update progress bar
+    job.percent = i / dbcount
+    ot[i]:attach_tag(nt)
+    count = count + 1
+  end
+
+  -- Delete old tag, this removes it from all images
+  darktable.tags.delete (ot)
+
+  job.valid = false
+  darktable.print ("Renamed tags for " .. count .. " images")
+  old_tag.editable = true
+  new_tag.editable = true
+
+  -- reset the gui fields
+
+  rename_reset()
+end
+
+-- GUI
+local old_widget = darktable.new_widget ("box") {
+    orientation = "horizontal",
+    darktable.new_widget("label") { label = "Old tag" },
+    old_tag
+}
+
+local new_widget = darktable.new_widget ("box") {
+    orientation = "horizontal",
+    darktable.new_widget("label") { label = "New tag" },
+    new_tag
+}
+
+local rename_widget = darktable.new_widget ("box") {
+    orientation = "vertical",
+    reset_callback = rename_reset,
+    old_widget,
+    new_widget,
+    darktable.new_widget("button") { label = "Go", clicked_callback = rename_tags }
+}
+
+
+darktable.register_lib ("rename_tags", "rename tag", true, true, {[darktable.gui.views.lighttable] = {"DT_UI_CONTAINER_PANEL_RIGHT_CENTER", 20},}, rename_widget, nil, nil)
+

contrib/select_untagged.lua

@@ -0,0 +1,65 @@
+--[[
+    This file is part of darktable,
+    copyright (c) 2017 Jannis_V
+
+    darktable is free software: you can redistribute it and/or modify
+    it under the terms of the GNU General Public License as published by
+    the Free Software Foundation, either version 3 of the License, or
+    (at your option) any later version.
+    darktable is distributed in the hope that it will be useful,
+     but WITHOUT ANY WARRANTY; without even the implied warranty of
+    MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+    GNU General Public License for more details.
+    You should have received a copy of the GNU General Public License
+    along with darktable.  If not, see <http://www.gnu.org/licenses/>.
+]]
+
+--[[
+Enable selection of untagged images (darktable|* tags are ignored)
+]]
+
+local dt = require "darktable"
+local du = require "lib/dtutils"
+local gettext = dt.gettext
+
+du.check_min_api_version("3.0.0", "select_untagged") 
+
+-- Tell gettext where to find the .mo file translating messages for a particular domain
+gettext.bindtextdomain("select_untagged",dt.configuration.config_dir.."/lua/locale/")
+
+local function _(msgid)
+  return gettext.dgettext("select_untagged", msgid)
+end
+
+local function stop_job(job)
+  job.valid = false
+end
+
+local function select_untagged_images()
+  job = dt.gui.create_job(_("select untagged images"), true, stop_job)
+
+  local selection = {}
+
+  for key,image in ipairs(dt.collection) do
+    if(job.valid) then
+      job.percent = (key-1)/#dt.collection
+      local tags =  dt.tags.get_tags(image)
+      local hasTags = false
+      for _,tag in ipairs(tags) do
+        if not string.match(tag.name,"darktable|") then
+          hasTags = true
+        end
+      end
+      if hasTags == false then
+        table.insert(selection,image)
+      end
+    else
+      break
+    end
+  end
+
+  job.valid = false
+  dt.gui.selection(selection)
+end
+
+dt.gui.libs.select.register_selection(_("select untagged"),select_untagged_images,_("select all images containing no tags or only tags added by darktable"))

contrib/slideshowMusic.lua

@@ -25,38 +25,28 @@ USAGE
 ]]
 
 local dt = require "darktable"
+local du = require "lib/dtutils"
+local df = require "lib/dtutils.file"
 require "official/yield"
-
 local gettext = dt.gettext
-dt.configuration.check_version(...,{2,0,2},{3,0,0})
+
+du.check_min_api_version("2.0.2", "slideshowMusic") 
 
 -- Tell gettext where to find the .mo file translating messages for a particular domain
-gettext.bindtextdomain("slideshowMusic",dt.configuration.config_dir.."/lua/")
+gettext.bindtextdomain("slideshowMusic",dt.configuration.config_dir.."/lua/locale/")
 
 local function _(msgid)
     return gettext.dgettext("slideshowMusic", msgid)
 end
 
-local function checkIfBinExists(bin)
-  local handle = io.popen("which "..bin)
-  local result = handle:read()
-  local ret
-  handle:close()
-  if (not result) then
-    dt.print_error(bin.." not found")
-    ret = false
-  end
-  ret = true
-  return ret
-end
-
 local function playSlideshowMusic(_, old_view, new_view)
   local filename, playMusic
 
   filename = dt.preferences.read("slideshowMusic","SlideshowMusic","string")
   playMusic = dt.preferences.read("slideshowMusic","PlaySlideshowMusic","bool")
 
-  if not checkIfBinExists("rhythmbox-client") then
+  if not df.check_if_bin_exists("rhythmbox-client") then
+    dt.print_error(_("rhythmbox-client not found"))
     return
   end
 

contrib/video_ffmpeg.lua

@@ -0,0 +1,460 @@
+--[[Timelapse video plugin based on ffmpeg for darktable 2.4.X and 2.6.X
+
+  copyright (c) 2018, 2019  Dominik Markiewicz
+  
+  darktable is free software: you can redistribute it and/or modify
+  it under the terms of the GNU General Public License as published by
+  the Free Software Foundation, either version 3 of the License, or
+  (at your option) any later version.
+  
+  darktable is distributed in the hope that it will be useful,
+  but WITHOUT ANY WARRANTY; without even the implied warranty of
+  MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+  GNU General Public License for more details.
+  
+  You should have received a copy of the GNU General Public License
+  along with darktable.  If not, see <http://www.gnu.org/licenses/>.
+]]
+
+--[[About this plugin
+This plugin will add the new export module "video ffmpeg".
+   
+----REQUIRED SOFTWARE----
+ffmpeg
+
+----USAGE----
+1. Go to Lighttable 
+2. Select images you want to use as a video ffmpeg frames
+3. In image export module select "video ffmpeg"
+4. Configure you video settings
+5. Export
+
+----WARNING----
+This script has been tested under Linux only
+]]
+
+local dt = require "darktable"
+local du = require "lib/dtutils"
+local df = require "lib/dtutils.file"
+local dsys = require "lib/dtutils.system"
+local gettext = dt.gettext
+
+du.check_min_api_version("5.0.0")
+
+local MODULE_NAME = "video_ffmpeg"
+
+local PS = dt.configuration.running_os == "windows" and  "\\"  or  "/"
+gettext.bindtextdomain(MODULE_NAME, dt.configuration.config_dir..PS.."lua"..PS.."locale"..PS)
+
+local function _(msgid)
+  return gettext.dgettext(MODULE_NAME, msgid)
+end
+
+
+---- DECLARATIONS
+
+local resolutions = {
+  ["QVGA"] = {
+    ["label"] = "QVGA 320x240 (4:3)",
+    ["w"] = 320,
+    ["h"] = 240
+  },
+  ["HVGA"] = {
+    ["label"] = "HVGA 480x320 (3:2)",
+    ["w"] = 480,
+    ["h"] = 320
+  },
+  ["VGA"] = {
+    ["label"] = "VGA 640x480 (4:3)",
+    ["w"] = 640,
+    ["h"] = 480
+  },
+  ["HDTV 720p"] = {
+    ["label"] = "HDTV 720p 1280x720 (16:9)",
+    ["w"] = 1280,
+    ["h"] = 720
+  },
+  ["HDTV 1080p"] = {
+    ["label"] = "HDTV 1080p 1920x1080 (16:9)",
+    ["w"] = 1920,
+    ["h"] = 1080
+  },
+  ["Cinema TV"] = {
+    ["label"] = "Cinema TV 2560x1080 (21:9)",
+    ["w"] = 2560,
+    ["h"] = 1080
+  },
+  ["2K"] = {
+    ["label"] = "2K 2048x1152 (16:9)",
+    ["w"] = 2048,
+    ["h"] = 1152
+  },
+  ["4K"] = {
+    ["label"] = "4K 4096x2304 (16:9)",
+    ["w"] = 4096,
+    ["h"] = 2304
+  }
+}
+
+local framerates = {"15", "16", "23.98", "24", "25", "29,97", "30", "48", "50", "59.94", "60"}
+
+local formats = {
+  ["AVI"] = {
+    ["extension"] = "avi",
+    ["codecs"] = {"mpeg4", "h263p", "h264", "mpeg2video", "hevc", "vp9"}
+  },
+  ["Matroska"] = {
+    ["extension"] = "mkv",
+    ["codecs"] = {"h264", "h263p", "mpeg4", "mpeg2video", "hevc", "vp9"}
+  },
+  ["WebM"] = {
+    ["extension"] = "webm",
+    ["codecs"] = {"vp9", "h263p", "h264", "mpeg4", "mpeg2video", "hevc"}
+  },
+  ["MP4"] = {
+    ["extension"] = "mp4",
+    ["codecs"] = {"h264", "h263p", "mpeg4", "mpeg2video"}
+  },
+  ["QuickTime"] = {
+    ["extension"] = "mov",
+    ["codecs"] = {"h264", "h263p", "mpeg4", "mpeg2video"}
+  }
+}
+
+local res_list = {}
+for i, v in pairs(resolutions) do
+  table.insert(res_list, v["label"])
+end
+
+table.sort(res_list)
+
+-- inital fulfill list of all available formats 
+local format_list = {}
+for i,v in pairs(formats) do
+  table.insert(format_list, i)
+end
+table.sort(format_list)
+
+-- initial fulfill list of all available codecs for the first (default if nothig saved yet) format
+-- if format was changed and stored in preferences, this list will be replaced by one matching selected format
+local codec_list = formats[format_list[1]]["codecs"]
+table.sort(codec_list)
+
+local function extract_resolution(description)
+  for _, v in pairs(resolutions) do
+    if v["label"] == description then
+      return v["w"].."x"..v["h"]
+    end
+  end
+  return "100x100"
+end
+
+---- GENERIC UTILS
+
+local function replace_combobox_elements(combobox, new_items, to_select)
+  if to_select == nil then
+    to_select = combobox.value
+  end
+
+  to_select_idx = 1
+  for i , name in ipairs(new_items) do
+    if name == to_select then
+      to_select_idx = i
+      break
+    end
+  end
+
+  local old_elements_count = #combobox
+  for i, name in ipairs(new_items) do
+    combobox[i] = name
+  end
+  if old_elements_count > #new_items then
+    for j = old_elements_count, #new_items + 1, -1 do
+      combobox[j] = nil
+    end
+  end
+
+  combobox.value = to_select_idx
+end
+
+--[[
+This function allow to format string by substitute tokens like `{label}` by value provided in `symbols` table
+  ex:
+```lua
+  format_string("I like {bananas} and {potatos}. {bananas}!", {["bananas"]: "darktable", ["potatos"]: "lua"}) 
+  -> "I like darktable and lua. darktable!"
+```
+
+I you want to preserve given currly braces, you need to escape it by backslash like so:
+```lua
+  format_string("it preserve \\{label\\} but substitute {label}", {["label"]: "this"}) 
+  -> "It preserve {label} but substitute "
+```
+--]]
+local function format_string(label, symbols)
+  local es1, es2 = "\u{ffe0}", "\u{ffe1}" -- for simplicity, just some strange utf characters 
+  local result = label:gsub("\\{", es1):gsub("\\}", es2)
+  for s,v in pairs(symbols) do
+    result = result:gsub("{"..s.."}", v)
+  end
+  return result:gsub(es1, "{"):gsub(es2, "}")
+end
+
+-----  COMPONENTS
+
+local function combobox_pref_read(name, all_values)
+  local value = dt.preferences.read(MODULE_NAME, name, "string")
+  for i,v in pairs(all_values) do
+    if v == value then return i end
+  end
+  return 1
+end
+
+local function combobox_pref_write(name)
+  local writer = function(widget)
+    dt.preferences.write(MODULE_NAME, name, "string", widget.value)
+  end
+  return writer
+end
+
+local function string_pref_read(name, default)
+  local value = dt.preferences.read(MODULE_NAME, name, "string")
+  if value ~= nil and value ~= "" then return value end
+  return default
+end
+
+local function string_pref_write(name, widget_attribute)
+  widget_attribute = widget_attribute or "value"
+  local writer = function(widget)
+    dt.preferences.write(MODULE_NAME, name, "string", widget[widget_attribute])
+  end
+  return writer
+end
+
+local framerates_selector = dt.new_widget("combobox"){
+  label = _("framerate"),
+  tooltip = _("select framerate of output video"),
+  value = combobox_pref_read("framerate", framerates),
+  changed_callback = combobox_pref_write("framerate"), 
+  table.unpack(framerates)
+}
+
+local res_selector = dt.new_widget("combobox"){
+  label = _("resolution"),
+  tooltip = _("select resolution of output video"),
+  value = combobox_pref_read("resolution", res_list),
+  changed_callback = combobox_pref_write("resolution"),
+  table.unpack(res_list)
+}
+
+local codec_selector = dt.new_widget("combobox"){
+  label = _("codec"),
+  tooltip = _("select codec"),
+  value = combobox_pref_read("codec", codec_list),
+  changed_callback = combobox_pref_write("codec"),
+  table.unpack(codec_list)
+}
+
+local format_selector = dt.new_widget("combobox"){
+  label = _("format container"),
+  tooltip = _("select format of output video"),
+  value = combobox_pref_read("format", format_list),
+  changed_callback = function(widget)
+    combobox_pref_write("format")(widget)
+    codec_list = formats[widget.value]["codecs"]
+    table.sort(codec_list)
+    replace_combobox_elements(codec_selector, codec_list)
+  end,
+  table.unpack(format_list)
+}
+
+local destination_label = dt.new_widget("section_label"){
+  label = _("output file destination"),
+  tooltip = _("settings of output file destination and name")
+}
+
+local defaultVideoDir = ''
+if dt.configuration.running_os == "windows" then
+  defaultVideoDir = os.getenv("USERPROFILE")..PS .."videos"
+elseif dt.configuration.running_os == "macos" then
+  defaultVideoDir =  os.getenv("home")..PS.."Videos"
+else
+  local handle = io.popen("xdg-user-dir VIDEOS")
+  defaultVideoDir = handle:read()
+  handle:close()
+end
+
+local output_directory_chooser = dt.new_widget("file_chooser_button"){
+  title = _("Select export path"),
+  is_directory = true,
+  tooltip =_("select the target directory for the timelapse. \nthe filename is created automatically."),
+  value = string_pref_read("export_path", defaultVideoDir),
+  changed_callback = string_pref_write("export_path")
+}
+
+local auto_output_directory_btn = dt.new_widget("check_button") {
+  label = "",
+  tooltip = _("if selected, output video will be placed in the same directory as first of selected images"),
+  value = not dt.preferences.read(MODULE_NAME, "not_auto_output_directory", "bool"), -- reverse, for true as default
+  clicked_callback = function (widget)
+    dt.preferences.write(MODULE_NAME, "not_auto_output_directory", "bool",  not widget.value)
+    output_directory_chooser.sensitive = not output_directory_chooser.sensitive 
+  end
+}
+
+local destination_box = dt.new_widget("box") {
+  orientation = "horizontal",
+  auto_output_directory_btn,
+  output_directory_chooser
+}
+
+local override_output_cb = dt.new_widget("check_button"){
+  label = _("override output file on conflict"),
+  tooltip = _("if checked, in case of file name conflict, the file will be overwritten"),
+  value = dt.preferences.read(MODULE_NAME, "override_output", "bool"),
+  clicked_callback = function (widget)
+    dt.preferences.write(MODULE_NAME, "override_output", "bool",  widget.value)
+  end
+}
+
+local filename_entry = dt.new_widget("entry"){
+  tooltip = _("enter output file name without extension.\n\n".. 
+    "You can use some placeholders:\n"..
+    "- {time} - time in format HH-mm-ss\n"..
+    "- {date} - date in foramt YYYY-mm-dd\n"..
+    "- {first_file} - name of first input file\n"..
+    "- {last_file} - name of last last_file"
+    ),
+  text = string_pref_read("filename_entry","timelapse_{date}_{time}"),
+ -- changed_callback = string_pref_write("filename_entry", "text") -- Not imlemented yet
+}
+
+local output_box = dt.new_widget("box"){
+  orientation="vertical",
+  destination_label,
+  override_output_cb,
+  destination_box,
+  filename_entry,
+}
+
+local open_after_export_cb = dt.new_widget("check_button"){
+  label = _(" open after export"),
+  tooltip = _("open video file after successful export"),
+  value = dt.preferences.read(MODULE_NAME, "open_after_export", "bool"),
+  clicked_callback = function (widget)
+    dt.preferences.write(MODULE_NAME, "open_after_export", "bool",  widget.value)
+  end
+}
+
+local widgets_list = {}
+if not df.check_if_bin_exists("ffmpeg") then
+  table.insert(widgets_list, df.executable_path_widget({"ffmpeg"}))
+end
+table.insert(widgets_list, res_selector)
+table.insert(widgets_list, framerates_selector)
+table.insert(widgets_list, format_selector)
+table.insert(widgets_list, codec_selector)
+table.insert(widgets_list, output_box)
+table.insert(widgets_list, open_after_export_cb)
+
+
+local module_widget = dt.new_widget("box") {
+  orientation = "vertical",
+  table.unpack(widgets_list)
+}
+
+---- EXPORT & REGISTRATION
+
+local function show_status(enf_storage, image, format, filename, number, total, high_quality, extra_data)
+  dt.print(_("export ")..tostring(number).." / "..tostring(total))   
+end
+
+local function init_export(storage, img_format, images, high_quality, extra_data)
+  -- store filename preference here cause there is no changed_callback on entry yet
+  string_pref_write("filename_entry", "text")(filename_entry)
+
+  extra_data["images"] = images -- needed, to preserve images order
+  extra_data["tmp_dir"] = dt.configuration.tmp_dir..PS..MODULE_NAME.."_"..os.time()
+  extra_data["fps"] = framerates_selector.value
+  extra_data["res"] = extract_resolution(res_selector.value)
+  extra_data["codec"] = codec_selector.value
+  extra_data["img_ext"] = "."..img_format.extension
+  local override_output = override_output_cb.value
+  local output_directory = auto_output_directory_btn.value and images[1].path or output_directory_chooser.value
+  local filename_mappings = {
+    date = os.date("%Y-%m-%d"),
+    time = os.date("%H-%M-%S"),
+    first_file = images[1].filename,
+    last_file = images[#images].filename
+  }
+  local output_extension = "."..formats[format_selector.value]["extension"]
+  local filename = format_string(filename_entry.text, filename_mappings)
+  local path = output_directory..PS..filename..output_extension
+  if not override_output then
+    path = df.create_unique_filename(path)
+  end
+
+  extra_data["output_file"] = path
+  extra_data["open_after_export"] = open_after_export_cb.value
+end
+
+local function export(extra_data)
+  local ffmpeg_path = df.check_if_bin_exists("ffmpeg")
+  if not ffmpeg_path then
+    dt.print_error("ffmpeg not found")
+    dt.print("ERROR - ffmpeg not found")
+    return
+  end
+  local dir = extra_data["tmp_dir"]
+  local fps = extra_data["fps"]
+  local res = extra_data["res"]
+  local codec = extra_data["codec"]
+  local img_ext = extra_data["img_ext"]
+  local output_file = extra_data["output_file"]
+  
+  local dir_create_result = df.mkdir(df.sanitize_filename(df.get_path(output_file)))
+  if dir_create_result ~= 0 then return dir_create_result end
+
+  local cmd = ffmpeg_path.." -y -r "..fps.." -i "..dir..PS.."%d"..img_ext.." -s:v "..res.." -c:v "..codec.." -crf 18 -preset veryslow "..df.sanitize_filename(output_file)
+  return dsys.external_command(cmd), output_file
+end
+
+local function finalize_export(storage, images_table, extra_data)
+    local tmp_dir = extra_data["tmp_dir"]
+    
+    dt.print(_("prepare merge process"))
+    
+    local result = df.mkdir(df.sanitize_filename(tmp_dir))
+    if result ~= 0 then dt.print(_("ERROR: cannot create temp directory")) end
+    
+    local images = extra_data["images"]
+    -- rename all images to consecutive numbers
+    for i, file in pairs(images) do
+      local filename = images_table[file]
+      dt.print_error(filename, file.filename)
+      df.file_move(filename, tmp_dir .. PS .. i .. extra_data["img_ext"])
+    end
+    dt.print("Start video building...")
+    local result, path = export(extra_data)
+    if result ~= 0 then 
+      dt.print(_("ERROR: cannot build image, see console for more info")) 
+    else
+      dt.print(_("SUCCESS"))
+      if extra_data["open_after_export"] then
+        dsys.launch_default_app(df.sanitize_filename(path))
+      end
+    end
+
+    df.rmdir(df.sanitize_filename(tmp_dir))
+end
+
+dt.register_storage(
+  "module_video_ffmpeg", 
+  _(MODULE_NAME), 
+  show_status, 
+  finalize_export,
+  nil, 
+  init_export, 
+  module_widget
+)
+

contrib/video_mencoder.lua

@@ -31,38 +31,39 @@ USAGE
 ]]
 
 local dt = require "darktable"
+local du = require "lib/dtutils"
+local df = require "lib/dtutils.file"
 require "official/yield"
+local gettext = dt.gettext
 
-dt.configuration.check_version(...,{2,0,1})
+du.check_min_api_version("2.0.1", "video_mencoder") 
+
+-- Tell gettext where to find the .mo file translating messages for a particular domain
+gettext.bindtextdomain("video_mencoder",dt.configuration.config_dir.."/lua/locale/")
+
+local function _(msgid)
+    return gettext.dgettext("video_mencoder", msgid)
+end
 
 local function show_status(storage, image, format, filename, number, total, high_quality, extra_data)
     dt.print("Export Image "..tostring(number).."/"..tostring(total))
 end
 
-local function checkIfBinExists(bin)
-    local handle = io.popen("which "..bin)
-    local result = handle:read()
-    local ret
-    handle:close()
-    if (result) then
-        --dt.print_error("true checkIfBinExists: "..bin)
-        ret = true
-    else
-        dt.print_error(bin.." not found")
-        ret = false
-    end
-  
-    return ret
-end
-
 local function create_video_mencoder(storage, image_table, extra_data)
-    if not checkIfBinExists("mencoder") then
+    dt.print("video_mencoder is deprecated.  Please use video_ffmpeg instead.  Please see the log for more details.")
+    dt.print_log("The mencoder executable is no longer distributed as part of most distributions,")
+    dt.print_log("therefore video_mencoder is deprecated. video_ffmpeg should be used in it's place.  video_mencoder")
+    dt.print_log("will be removed when darktable 2.8 is released.")
+    if not df.check_if_bin_exists("mencoder") then
+        dt.print_error(_("mencoder not found"))
         return
     end
-    if not checkIfBinExists("xdg-open") then
+    if not df.check_if_bin_exists("xdg-open") then
+        dt.print_error(_("xdg-open not found"))
         return
     end
-    if not checkIfBinExists("xdg-user-dir") then
+    if not df.check_if_bin_exists("xdg-user-dir") then
+        dt.print_error(_("xdg-user-dir not found"))
         return
     end
 
@@ -111,6 +112,9 @@ dt.preferences.register("video_mencoder", "OpenVideo", "bool", "Video exort (MEn
 local handle = io.popen("xdg-user-dir VIDEOS")
 local result = handle:read()
 handle:close()
+if (result == nil) then
+	result = ""
+end
 dt.preferences.register("video_mencoder", "ExportDirectory", "directory", "Video exort (MEncoder): Video export directory","A directory that will be used to export a Video",result)
 
 -- Get the MEncoder codec list with: mencoder -ovc help

examples/gettextExample.lua

@@ -53,7 +53,10 @@ LUA ERROR Hallo Welt!
 
 ]] 
 local dt = require "darktable"
-dt.configuration.check_version(...,{3,0,0})
+local du = require "lib/dtutils"
+
+--check API version
+du.check_min_api_version("3.0.0", "gettextExample")
 
 -- Not translated Text
 dt.print_error("Hello World!")
@@ -64,7 +67,7 @@ dt.print_error(gettext.gettext("image"))
 
 -- Tell gettext where to find the .mo file translating messages for a particular domain
 
-gettext.bindtextdomain("gettextExample",dt.configuration.config_dir.."/lua/")
+gettext.bindtextdomain("gettextExample",dt.configuration.config_dir.."/lua/locale/")
 -- Translate a string using the specified textdomain
 dt.print_error(gettext.dgettext("gettextExample", 'Hello World!'))
 

examples/hello_world.lua

@@ -27,7 +27,9 @@ USAGE
 
 ]]
 local dt = require "darktable"
-dt.configuration.check_version(...,{2,0,0},{3,0,0})
+local du = require "lib/dtutils"
+
+du.check_min_api_version("2.0.0", "hello_world") 
 
 dt.print("hello, world")
 

examples/moduleExample.lua

@@ -31,8 +31,9 @@ https://www.darktable.org/lua-api/index.html.php#darktable_new_widget
 ]]
 
 local dt = require "darktable"
+local du = require "lib/dtutils"
 
-dt.configuration.check_version(...,{3,0,0})
+du.check_min_api_version("3.0.0", "moduleExample") 
 
 -- add a new lib
 local check_button = dt.new_widget("check_button"){label = "MyCheck_button", value = true}
@@ -70,33 +71,69 @@ local slider = dt.new_widget("slider")
   value = 52          -- The current value of the slider
 }
 
-dt.register_lib(
-  "exampleModule",     -- Module name
-  "exampleModule",     -- name
-  true,                -- expandable
-  false,               -- resetable
-  {[dt.gui.views.lighttable] = {"DT_UI_CONTAINER_PANEL_RIGHT_CENTER", 100}},   -- containers
-  dt.new_widget("box") -- widget
+if (dt.configuration.api_version_major >= 6) then
+  local section_label = dt.new_widget("section_label")
   {
-    orientation = "vertical",
-    dt.new_widget("button")
+    label = "MySectionLabel"
+  }
+
+  dt.register_lib(
+    "exampleModule",     -- Module name
+    "exampleModule",     -- name
+    true,                -- expandable
+    false,               -- resetable
+    {[dt.gui.views.lighttable] = {"DT_UI_CONTAINER_PANEL_RIGHT_CENTER", 100}},   -- containers
+    dt.new_widget("box") -- widget
     {
-      label = "MyButton",
-      clicked_callback = function (_)
-        dt.print("Button clicked")
-      end
+      orientation = "vertical",
+      dt.new_widget("button")
+      {
+        label = "MyButton",
+        clicked_callback = function (_)
+          dt.print("Button clicked")
+        end
+      },
+      check_button,
+      combobox,   
+      entry,
+      file_chooser_button,
+      label,
+      separator,
+      slider,
+	  section_label
     },
-    check_button,
-    combobox,   
-    entry,
-    file_chooser_button,
-    label,
-    separator,
-    slider
-  },
-  nil,-- view_enter
-  nil -- view_leave
-)
+    nil,-- view_enter
+    nil -- view_leave
+  )
+else
+  dt.register_lib(
+    "exampleModule",     -- Module name
+    "exampleModule",     -- name
+    true,                -- expandable
+    false,               -- resetable
+    {[dt.gui.views.lighttable] = {"DT_UI_CONTAINER_PANEL_RIGHT_CENTER", 100}},   -- containers
+    dt.new_widget("box") -- widget
+    {
+      orientation = "vertical",
+      dt.new_widget("button")
+      {
+        label = "MyButton",
+        clicked_callback = function (_)
+          dt.print("Button clicked")
+        end
+      },
+      check_button,
+      combobox,   
+      entry,
+      file_chooser_button,
+      label,
+      separator,
+      slider
+    },
+    nil,-- view_enter
+    nil -- view_leave
+  )
+end
 
 -- vim: shiftwidth=2 expandtab tabstop=2 cindent syntax=lua
 -- kate: hl Lua;

examples/multi_os.lua

@@ -0,0 +1,244 @@
+--[[
+    This file is part of darktable,
+    copyright (c) 2017,2018 Bill Ferguson <wpferguson@gmail.com>
+
+    darktable is free software: you can redistribute it and/or modify
+    it under the terms of the GNU General Public License as published by
+    the Free Software Foundation, either version 3 of the License, or
+    (at your option) any later version.
+
+    darktable is distributed in the hope that it will be useful,
+     but WITHOUT ANY WARRANTY; without even the implied warranty of
+    MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+    GNU General Public License for more details.
+
+    You should have received a copy of the GNU General Public License
+    along with darktable.  If not, see <http://www.gnu.org/licenses/>.
+]]
+--[[
+    multi_os - an example script that runs on linux, MacOS, and Windows.
+
+    multi_os is an example of how to write a script that will run on different
+    operating systems.  It uses the lua-scripts libraries to lessen the amount
+    of code that needs to be written, as well as gaining access to tested 
+    cross-platform routines.  This script also performs a function that some 
+    may find useful.  It creates a button in the lighttable selected images module
+    that extracts the embedded jpeg image from a raw file, then imports it and groups
+    it with the raw file.  A keyboard shortcut is also created.  A key combination can
+    be assigned to the shortcut in the lua preferences and then the action can be invoked
+    by hovering over the image and pressing the key combination.
+
+    ADDITIONAL SOFTWARE NEEDED FOR THIS SCRIPT
+    * ufraw-batch - https://ufraw.sourceforge.net
+                    MacOS - install with homebrew
+
+    USAGE
+    * require this script from your main lua file
+    * start darktable, open prefreences, go to lua options 
+      and update the executable location if you are running
+      Windows or MacOS, then restart darktable.
+    * select an image or images
+    * click the button to extract the jpeg
+
+    BUGS, COMMENTS, SUGGESTIONS
+    * Send to Bill Ferguson, wpferguson@gmail.com
+
+    CHANGES
+]]
+
+--[[
+    require "darktable" provides the interface to the darktable lua functions used to
+    interact with darktable
+]]
+
+local dt = require "darktable"
+
+--[[
+    require "lib/..." provides access to functions that have been pulled from various 
+    scripts and consolidated into libraries.  Using the libraries eliminates having to 
+    recode common functions in every script.
+]]
+
+local du = require "lib/dtutils"  -- utilities
+local df = require "lib/dtutils.file"   -- file utilities
+local dtsys = require "lib/dtutils.system"  -- system utilities
+
+--[[
+    darktable is an international program, and it's user interface has been translated into
+    many languages.  The lua API provides gettext which is a function that looks for and replaces
+    strings with the translated equivalents based on locale.  Even if you don't provide the 
+    translations, inserting this lays the groundwork for anyone who wants to translate the strings.
+]]
+
+local gettext = dt.gettext
+
+-- Tell gettext where to find the .mo file translating messages for a particular domain
+gettext.bindtextdomain("multi_os",dt.configuration.config_dir.."/lua/locale/")
+
+local function _(msgid)
+    return gettext.dgettext("multi_os", msgid)
+end
+
+--[[
+    Check that the current api version is greater than or equal to the specified minimum.  If it's not
+    then du.check_min_api_version will print an error to the log and return false.  If the minimum api is
+    not met, then just refuse to load and return.  Optionally, you could print an error message to the 
+    screen stating that you couldn't load because the minimum api version wasn't met.
+]]
+
+du.check_min_api_version("5.0.0", "multi_os") 
+
+--[[
+    copy_image_attributes is a local subroutine to copy image attributes in the database from the raw image
+    to the extracted jpeg image.  Even though the subroutine could act on the variables in the main program 
+    directly, passing them as arguments and using them in the subroutine is much safer.  
+]]
+
+local function copy_image_attributes(from, to, ...)
+  local args = {...}
+  if #args == 0 then
+    args[1] = "all"
+  end
+  if args[1] == "all" then
+    args[1] = "rating"
+    args[2] = "colors"
+    args[3] = "exif"
+    args[4] = "meta"
+    args[5] = "GPS"
+  end
+  for _,arg in ipairs(args) do
+    if arg == "rating" then
+      to.rating = from.rating
+    elseif arg == "colors" then
+      to.red = from.red
+      to.blue = from.blue
+      to.green = from.green
+      to.yellow = from.yellow
+      to.purple = from.purple
+    elseif arg == "exif" then
+      to.exif_maker = from.exif_maker
+      to.exif_model = from.exif_model
+      to.exif_lens = from.exif_lens
+      to.exif_aperture = from.exif_aperture
+      to.exif_exposure = from.exif_exposure
+      to.exif_focal_length = from.exif_focal_length
+      to.exif_iso = from.exif_iso
+      to.exif_datetime_taken = from.exif_datetime_taken
+      to.exif_focus_distance = from.exif_focus_distance
+      to.exif_crop = from.exif_crop
+    elseif arg == "GPS" then
+      to.elevation = from.elevation
+      to.longitude = from.longitude
+      to.latitude = from.latitude
+    elseif arg == "meta" then
+      to.publisher = from.publisher
+      to.title = from.title
+      to.creator = from.creator
+      to.rights = from.rights
+      to.description = from.description
+    else
+      dt.print_error("Unrecognized option to copy_image_attributes: " .. arg)
+    end
+  end
+end
+
+--[[
+    The main function called from the button or the shortcut.  It takes one or more raw files, passed
+    in a table and extracts the embedded jpeg images
+]]
+
+local function extract_embedded_jpeg(images)
+
+  --[[
+      check if the executable exists, since we can't do anything without it.  check_if_bin_exists() 
+      checks to see if there is a saved  executable location.  If not, it checks for the executable 
+      in the user's path.  When it finds the executable, it returns the command to run it.
+  ]]
+
+  local ufraw_executable = df.check_if_bin_exists("ufraw-batch")
+  if ufraw_executable then
+    for _, image in ipairs(images) do
+      if image.is_raw then
+        local img_file = du.join({image.path, image.filename}, "/")
+
+        --[[
+            dtsys.external_command() is operating system aware and formats the command as necessary.  
+            df.sanitize_filename() is used to quote the image filepath to protect from spaces.  It is also
+            operating system aware and uses the corresponding quotes.
+        ]]
+
+        if dtsys.external_command(ufraw_executable .. " --silent --embedded-image " .. df.sanitize_filename(img_file)) then
+          local jpg_img_file = df.chop_filetype(img_file) .. ".embedded.jpg"
+          dt.print_log("jpg_img_file set to ", jpg_img_file) -- print a debugging message
+          local myimage = dt.database.import(jpg_img_file)
+          myimage:group_with(image.group_leader)
+          copy_image_attributes(image, myimage, "all")
+
+          --[[
+              copy all of the tags except the darktable tags
+          ]]
+
+          for _,tag in pairs(dt.tags.get_tags(image)) do 
+            if not (string.sub(tag.name,1,9) == "darktable") then
+              dt.print_log("attaching tag")
+              dt.tags.attach(tag,myimage)
+            end
+          end
+        end
+      else
+        dt.print_error(image.filename .. " is not a raw file.  No image can be extracted") -- print debugging error message
+        dt.print(image.filename .. " is not a raw file.  No image can be extracted") -- print the error to the screen
+      end
+    end
+  else
+    dt.print_error("ufraw-batch not found.  Exiting...") -- print debugging error message
+    dt.print("ufraw-batch not found.  Exiting...") -- print the error to the screen
+  end
+end
+
+--[[
+    Windows and MacOS don't place executables in the user's path so their location needs to be specified
+    so that the script can find them. An exception to this is packages installed on MacOS with homebrew.  Those
+    executables are put in /usr/local/bin.  These are saved as executable path preferences.  check_if_bin_exists()
+    looks for the executable path preference for the executable.  If it doesn't find one, then the path is checked 
+    to see if the executable is there.
+]]
+
+local executable = "ufraw-batch"
+local ufraw_batch_path_widget = dt.new_widget("file_chooser_button"){
+  title = _("Select ufraw-batch[.exe] executable"),
+  value = df.get_executable_path_preference(executable),
+  is_directory = false,
+  changed_callback = function(self)
+    if df.check_if_bin_exists(self.value) then
+      df.set_executable_path_preference(executable, self.value)
+    end
+  end
+}
+dt.preferences.register("executable_paths", "ufraw-batch", -- name
+  "file", -- type
+  _('multi_os: ufraw-batch location'),  -- label
+  _('Installed location of ufraw-batch. Requires restart to take effect.'), -- tooltip
+  "ufraw-batch", -- default
+  ufraw_batch_path_widget
+)
+
+--[[
+    Add a button to the selected images module in lighttable
+]]
+
+dt.gui.libs.image.register_action(
+  _("extract embedded jpeg"),
+  function(event, images) extract_embedded_jpeg(images) end,
+  "extract embedded jpeg"
+)
+
+--[[
+    Add a shortcut
+]]
+
+dt.register_event(
+  "shortcut",
+  function(event, shortcut) extract_embedded_jpeg(dt.gui.action_images) end,
+  "extract embedded jpeg"
+)

examples/preferenceExamples.lua

@@ -22,7 +22,9 @@ USAGE
 * require this script from your main lua file
 ]] 
 local dt = require "darktable"
-dt.configuration.check_version(...,{2,0,1},{3,0,0})
+local du = require "lib/dtutils"
+
+du.check_min_api_version("2.0.1", "preferenceExamples") 
 
 dt.preferences.register("preferenceExamples",        -- script: This is a string used to avoid name collision in preferences (i.e namespace). Set it to something unique, usually the name of the script handling the preference.
                         "preferenceExamplesString",  -- name

examples/printExamples.lua

@@ -0,0 +1,42 @@
+--[[
+    This file is part of darktable,
+    copyright (c) 2017 Tobias Jakobs
+    darktable is free software: you can redistribute it and/or modify
+    it under the terms of the GNU General Public License as published by
+    the Free Software Foundation, either version 3 of the License, or
+    (at your option) any later version.
+    darktable is distributed in the hope that it will be useful,
+    but WITHOUT ANY WARRANTY; without even the implied warranty of
+    MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+    GNU General Public License for more details.
+    You should have received a copy of the GNU General Public License
+    along with darktable.  If not, see <http://www.gnu.org/licenses/>.
+]]
+
+--[[
+Print Examples
+prints "hello world when DT starts
+USAGE
+* require this file from your main lua config file:
+]]
+local dt = require "darktable"
+local du = require "lib/dtutils"
+
+du.check_min_api_version("5.0.0", "printExamples") 
+
+-- Will print a string to the darktable control log (the long
+-- overlayed window that appears over the main panel).
+dt.print("print")
+
+-- This function will print its parameter if the Lua logdomain is
+-- activated. Start darktable with the "-d lua" command line option
+-- to enable the Lua logdomain.
+dt.print_error("print error")
+
+-- This function will print its parameter if the Lua logdomain is
+-- activated. Start darktable with the "-d lua" command line option
+-- to enable the Lua logdomain.
+dt.print_log("print log")
+
+--
+-- vim: shiftwidth=2 expandtab tabstop=2 cindent syntax=lua

examples/running_os.lua

@@ -0,0 +1,36 @@
+--[[
+    This file is part of darktable,
+    copyright (c) 2017 Tobias Jakobs
+
+    darktable is free software: you can redistribute it and/or modify
+    it under the terms of the GNU General Public License as published by
+    the Free Software Foundation, either version 3 of the License, or
+    (at your option) any later version.
+
+    darktable is distributed in the hope that it will be useful,
+     but WITHOUT ANY WARRANTY; without even the implied warranty of
+    MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+    GNU General Public License for more details.
+
+    You should have received a copy of the GNU General Public License
+    along with darktable.  If not, see <http://www.gnu.org/licenses/>.
+]]
+--[[
+
+USAGE
+* require this script from your main lua file
+  To do this add this line to the file .config/darktable/luarc: 
+require "running_os"
+
+prints the operating system
+
+]]
+local dt = require "darktable"
+local du = require "lib/dtutils"
+
+du.check_min_api_version("5.0.0", "running_os") 
+
+dt.print("You are running: "..dt.configuration.running_os)
+
+--
+-- vim: shiftwidth=2 expandtab tabstop=2 cindent syntax=lua

include_all.lua

@@ -1,6 +1,7 @@
 --[[
   This file is part of darktable,
   copyright (c) 2014 Jérémy Rosen
+  copyright (c) 2018 Bill Ferguson
   
   darktable is free software: you can redistribute it and/or modify
   it under the terms of the GNU General Public License as published by
@@ -33,14 +34,20 @@ Note that you need to restart DT for your changes to enabled scripts to take eff
 ]]
 local dt = require "darktable"
 local io = require "io"
-dt.configuration.check_version(...,{3,0,0})
 
-local output = io.popen("cd "..dt.configuration.config_dir.."/lua ;find . -name \\*.lua -print")
+-- must be loaded for scripts using darktable.control_execute to work
+require "official/yield"
+
+dt.configuration.check_version(...,{3,0,0},{4,0,0},{5,0,0})
+
+-- find all scripts, but skip the lib and tools directories
+local output = io.popen("cd "..dt.configuration.config_dir.."/lua ;find . -name lib -prune -o -name tools -prune -o -name \\*.lua -print")
+
 local my_name={...}
 my_name = my_name[1]
 for line in output:lines() do
   local req_name = line:sub(3,-5)
-  if req_name ~= my_name then
+  if req_name ~= my_name and not string.match(req_name, "yield") then
     dt.preferences.register(my_name,req_name,"bool","enable "..req_name,
     "Should the script "..req_name.." be enabled at next startup",false)
 

lib/dtutils.lua

@@ -32,7 +32,42 @@ local dt = require "darktable"
 
 local log = require "lib/dtutils.log"
 
-dt.configuration.check_version(...,{3,0,0})
+dtutils.libdoc.functions["check_min_api_version"] = {
+  Name = [[check_min_api_version]],
+  Synopsis = [[check the minimum required api version against the current api version]],
+  Usage = [[local du = require "lib/dtutils"
+
+    local result = du.check_min_api_version(min_api, script_name)
+      min_api - string - the api version that the application was written for (example: "5.0.0")
+      script_name - string - the name of the script]],
+  Description = [[check_min_api_version compares the minimum api required for the appllication to
+    run against the current api version. The minimum api version is typically the api version that 
+    was current when the application was created. If the minimum api version is not met, then an 
+    error message is printed saying the script_name failed to load, then an error is thrown causing the
+    program to stop executing. 
+
+    This function is intended to replace darktable.configuration.check_version(). The application code
+    won't have to be updated each time the api changes because this only checks the minimum version required.]],
+  Return_Value = [[result - true if the minimum api version is available, false if not.]],
+  Limitations = [[When using the default handler on a script being executed from the luarc file, the error thrown
+    will stop the luarc file from executing any remaining statements. This limitation does not apply to script_manger.]],
+  Example = [[check_min_api_version("5.0.0") does nothing if the api is greater or equal to 5.0.0 otherwise an
+    error message is printed and an error is thrown stopping execution of the script.]],
+  See_Also = [[]],
+  Reference = [[]],
+  License = [[]],
+  Copyright = [[]],
+}
+
+function dtutils.check_min_api_version(min_api, script_name)
+  local current_api = dt.configuration.api_version_string
+  if min_api > current_api then
+    dt.print_error("This application is written for lua api version " .. min_api .. " or later.")
+    dt.print_error("The current lua api version is " .. current_api)
+    dt.print("ERROR: " .. script_name .. " failed to load. Lua API version " .. min_api .. " or later required.")
+    error("Minimum API " .. min_api .. " not met for " .. script_name .. ".", 0)
+  end
+end
 
 dtutils.libdoc.functions["split"] = {
   Name = [[split]],
@@ -215,4 +250,37 @@ function dtutils.spairs(_table, order) -- Code copied from http://stackoverflow.
   end
 end
 
+dtutils.libdoc.functions["check_os"] = {
+  Name = [[check_os]],
+  Synopsis = [[check that the operating system is supported]],
+  Usage = [[local du = require "lib/dtutils"
+
+    local result = du.check_os(operating_systems)
+      operating_systems - a table of operating system names such as {"windows","linux","macos","unix"}]],
+  Description = [[check_os checks a supplied table of operating systems against the operating system the
+    script is running on and returns true if the OS is in the list, otherwise false]],
+  Return_Value = [[result - boolean - true if the operating system is supported, false if not.]],
+  Limitations = [[]],
+  Example = [[local du = require "lib/dtutils"
+              if du.check_os({"windows"}) then
+                -- run the script
+              else
+                dt.print("Script <script name> only runs on windows")
+                return
+              end]],
+  See_Also = [[]],
+  Reference = [[]],
+  License = [[]],
+  Copyright = [[]],
+}
+
+function dtutils.check_os(operating_systems)
+  for _,os in pairs(operating_systems) do
+    if dt.configuration.running_os == os then
+      return true
+    end
+  end
+  return false
+end
+
 return dtutils

lib/dtutils/file.lua

@@ -1,5 +1,8 @@
 local dtutils_file = {}
 local dt = require "darktable"
+local du = require "lib/dtutils"
+local ds = require "lib/dtutils.string"
+local dsys = require "lib/dtutils.system"
 
 local log = require "lib/dtutils.log"
 
@@ -23,7 +26,7 @@ dtutils_file.libdoc = {
 
 local gettext = dt.gettext
 
-dt.configuration.check_version(...,{3,0,0})
+du.check_min_api_version("5.0.0", "dtutils.file")
 
 -- Tell gettext where to find the .mo file translating messages for a particular domain
 gettext.bindtextdomain("dtutils.file",dt.configuration.config_dir.."/lua/locale/")
@@ -34,14 +37,18 @@ end
 
 dtutils_file.libdoc.functions["check_if_bin_exists"] = {
   Name = [[check_if_bin_exists]],
-  Synopsis = [[check if an executable is in the path]],
+  Synopsis = [[check if an executable exists]],
   Usage = [[local df = require "lib/dtutils.file"
 
     local result = df.check_if_bin_exists(bin)
       bin - string - the binary to check for]],
-  Description = [[check_if_bin_exists checks to see if the specified binary executable is
-    in the path.]],
-  Return_Value = [[result - boolean - true if the executable was found, false if not]],
+  Description = [[check_if_bin_exists checks to see if the specified binary exists.
+    check_if_bin_exists first checks to see if a preference for the binary has been
+    registered and uses that if found.  The presence of the file is verified, then 
+    quoted and returned.  If no preference is specified and the operating system is
+    linux then the which command is used to check for a binary in the path.  If found
+    that path is returned.  If no binary is found, false is returned.]],
+  Return_Value = [[result - string - the path of the binary, false if not found]],
   Limitations = [[]],
   Example = [[]],
   See_Also = [[]],
@@ -51,9 +58,30 @@ dtutils_file.libdoc.functions["check_if_bin_exists"] = {
 }
 
 function dtutils_file.check_if_bin_exists(bin)
-  local result = os.execute("which " .. bin)
-  if not result then
-    result = false
+  local result = false
+  local path = nil
+
+  if string.match(bin, "/") or string.match(bin, "\\") then 
+    path = bin
+  else
+    path = dtutils_file.get_executable_path_preference(bin)
+  end
+
+  if string.len(path) > 0 then
+    if dtutils_file.check_if_file_exists(path) then
+      if (string.match(path, ".exe$") or string.match(path, ".EXE$")) and dt.configuration.running_os ~= "windows" then
+       result = "wine " .. "\"" .. path .. "\""      
+      else
+        result = "\"" .. path .. "\""
+      end
+    end
+  elseif dt.configuration.running_os == "linux" then
+    local p = io.popen("which " .. bin)
+    local output = p:read("*a")
+    p:close()
+    if string.len(output) > 0 then
+      result = output:sub(1,-2)
+    end
   end
   return result
 end
@@ -194,10 +222,28 @@ dtutils_file.libdoc.functions["check_if_file_exists"] = {
 }
 
 function dtutils_file.check_if_file_exists(filepath)
-  local result = os.execute("test -e " .. filepath)
-  if not result then
-    result = false
+  local result
+  if (dt.configuration.running_os == 'windows') then
+    filepath = string.gsub(filepath, '[\\/]+', '\\')
+    result = os.execute('if exist "'..filepath..'" (cmd /c exit 0) else (cmd /c exit 1)')
+    if not result then
+      result = false
+    end
+  elseif (dt.configuration.running_os == "linux") then
+    result = os.execute('test -e ' .. "\"" .. filepath .. "\"")
+    if not result then
+      result = false
+    end
+  else
+    local file = io.open(filepath, "r")
+    if file then
+      result = true
+      file:close()
+    else
+      result = false
+    end
   end
+
   return result
 end
 
@@ -245,9 +291,12 @@ dtutils_file.libdoc.functions["file_copy"] = {
 function dtutils_file.file_copy(fromFile, toFile)
   local result = nil
   -- if cp exists, use it
-  if dtutils_file.check_if_bin_exists("cp") then
+  if dt.configuration.running_os == "windows" then
+    result = os.execute('copy "' .. fromFile .. '" "' .. toFile .. '"')
+  elseif dtutils_file.check_if_bin_exists("cp") then
     result = os.execute("cp '" .. fromFile .. "' '" .. toFile .. "'")
   end
+
   -- if cp was not present, or if cp failed, then a pure lua solution
   if not result then
     local fileIn, err = io.open(fromFile, 'rb')
@@ -384,11 +433,180 @@ function dtutils_file.create_unique_filename(filepath)
   while dtutils_file.check_if_file_exists(filepath) do
     filepath = dtutils_file.filename_increment(filepath)
     -- limit to 99 more exports of the original export
-    if string.match(dtfileutils.get_basename(filepath), "_(d-)$") == "99" then 
+    if string.match(dtutils_file.get_basename(filepath), "_(d-)$") == "99" then
       break 
     end
   end
   return filepath
 end
 
+
+dtutils_file.libdoc.functions["set_executable_path_preference"] = {
+  Name = [[set_executable_path_preference]],
+  Synopsis = [[set a preference for the path to an executable]],
+  Usage = [[local df = require "lib/dtutils.file"
+
+    df.set_executable_path_preference(executable, path)
+      executable - string - the name of the executable to set the path for
+      path - string - the path to the binary]],
+  Description = [[set_executable_path_preference takes an executable name and path to the
+  executable and registers the preference for later use.]],
+  Return_Value = [[]],
+  Limitations = [[]],
+  Example = [[]],
+  See_Also = [[]],
+  Reference = [[]],
+  License = [[]],
+  Copyright = [[]],
+}
+
+function dtutils_file.set_executable_path_preference(executable, path)
+  dt.preferences.write("executable_paths", executable, "string", path)
+end
+
+
+dtutils_file.libdoc.functions["get_executable_path_preference"] = {
+  Name = [[get_executable_path_preference]],
+  Synopsis = [[return the path to an executable from a preference]],
+  Usage = [[local df = require "lib/dtutils.file"
+
+    local result = df.get_executable_path_preference(executable)
+      executable - string - the name of the executable to get the path for]],
+  Description = [[get_executable_path_preference returns the path preference to
+    the requested executable.]],
+  Return_Value = [[result - string - path to the executable]],
+  Limitations = [[executable should be the basename of the executable without extensions]],
+  Example = [[]],
+  See_Also = [[]],
+  Reference = [[]],
+  License = [[]],
+  Copyright = [[]],
+}
+
+function dtutils_file.get_executable_path_preference(executable)
+  return dt.preferences.read("executable_paths", executable, "string")
+end
+
+
+dtutils_file.libdoc.functions["executable_path_widget"] = {
+  Name = [[executable_path_widget]],
+  Synopsis = [[create a widget to get executable path preferences]],
+  Usage = [[local df = require "lib/dtutils.file"
+
+    local widget = df.executable_path_widget(executables)
+      executables - table - a table of strings that are executable names]],
+  Description = [[executable_path_widget takes a table of executable names
+    and builds a set of file selector widgets to get the path to the executable. 
+    The resulting widgets are wrapped in a box widget and returned.]],
+  Return_Value = [[widget - widget - a widget containing a file selector widget for
+    each executable.]],
+  Limitations = [[]],
+  Example = [[]],
+  See_Also = [[]],
+  Reference = [[]],
+  License = [[]],
+  Copyright = [[]],
+}
+
+function dtutils_file.executable_path_widget(executables)
+  local box_widgets = {}
+  table.insert(box_widgets, dt.new_widget("section_label"){label = "select executable(s)"})
+  for _, executable in pairs(executables) do 
+    table.insert(box_widgets, dt.new_widget("label"){label = "select " .. executable .. " executable"})
+    local path = dtutils_file.get_executable_path_preference(executable)
+    if not path then 
+      path = ""
+    end
+    table.insert(box_widgets, dt.new_widget("file_chooser_button"){
+      title = "select " .. executable .. " executable",
+      value = path,
+      is_directory = false,
+      changed_callback = function(self)
+        if dtutils_file.check_if_bin_exists(self.value) then
+          dtutils_file.set_executable_path_preference(executable, self.value)
+        end
+      end}
+    )
+  end
+  local box = dt.new_widget("box"){
+    orientation = "vertical",
+    table.unpack(box_widgets)
+  }
+  return box
+end
+
+dtutils_file.libdoc.functions["sanitize_filename"] = {
+  Name = [[sanitize_filename]],
+  Synopsis = [[make a filename safe to pass as an argument]],
+  Usage = [[local df = require "lib/dtutils.file"
+
+    local sanitized_filename = df.sanitize_filename(filename)
+      filename - string - a filepath and filename]],
+  Description = [[sanitize_file places quotes around the filename in an
+    operating system specific manner.  The result is safe to pass as 
+    an argument to the operating system.]],
+  Return_Value = [[sanitized_filename - string - quoted filename]],
+  Limitations = [[]],
+  Example = [[]],
+  See_Also = [[]],
+  Reference = [[]],
+  License = [[]],
+  Copyright = [[]],
+}
+
+function dtutils_file.sanitize_filename(filename)
+  return ds.sanitize(filename)
+end
+
+dtutils_file.libdoc.functions["mkdir"] = {
+  Name = [[mkdir]],
+  Synopsis = [[create the directory(ies) if they do not already exists]],
+  Usage = [[local df = require "lib/dtutils.file"
+
+     df.mkdir(path)
+      path - string - a directory path]],
+  Description = [[mkdir creates directories if not already exists. It 
+    create whole parents subtree if needed
+  ]],
+  Return_Value = [[path - string - a directory path]],
+  Limitations = [[]],
+  Example = [[]],
+  See_Also = [[]],
+  Reference = [[]],
+  License = [[]],
+  Copyright = [[]],
+}
+function dtutils_file.mkdir(path) 
+  if not dtutils_file.check_if_file_exists(path) then
+    local mkdir_cmd = dt.configuration.running_os == "windows" and "mkdir" or "mkdir -p"
+    return dsys.external_command(mkdir_cmd.." "..path)
+  else
+    return 0
+  end
+end
+
+dtutils_file.libdoc.functions["rmdir"] = {
+  Name = [[rmdir]],
+  Synopsis = [[recursively remove a directory]],
+  Usage = [[local df = require "lib/dtutils.file"
+
+     df.rmdir(path)
+      path - string - a directory path]],
+  Description = [[rm allow to recursively remove directories]],
+  Return_Value = [[path - string - a directory path]],
+  Limitations = [[]],
+  Example = [[]],
+  See_Also = [[]],
+  Reference = [[]],
+  License = [[]],
+  Copyright = [[]],
+}
+
+function dtutils_file.rmdir(path)
+  local rm_cmd = dt.configuration.running_os == "windows" and "rmdir /S /Q" or "rm -r"
+  return dsys.external_command(rm_cmd.." "..path)
+end
+
+
 return dtutils_file
+

lib/dtutils/log.lua

@@ -48,8 +48,7 @@ dtutils_log.libdoc = {
 
 local dt = require "darktable"
 local dt_print_error = dt.print_error
--- work around until dt.print_log()
-local dt_print_log = dt.print_error
+local dt_print_log = dt.print_log
 local dt_print = dt.print
 
 -- set the default log levels
@@ -243,6 +242,7 @@ function dtutils_log.log_level(...)
     for _,v in ipairs(levels) do
       if dtutils_log[v].enabled == true then
         log_level = dtutils_log[v]
+        break
       end
     end
   end

lib/dtutils/string.lua

@@ -1,4 +1,7 @@
 local dtutils_string = {}
+
+local dt = require "darktable"
+
 dtutils_string.libdoc = {
   Name = [[dtutils.string]],
   Synopsis = [[a library of string utilities for use in darktable lua scripts]],
@@ -170,4 +173,35 @@ function dtutils_string.urlencode(str)
   return str
 end
 
+
+dtutils_string.libdoc.functions["sanitize"] = {
+  Name = [[sanitize]],
+  Synopsis = [[surround a string in quotes making it safe to pass as an argument]],
+  Usage = [[local ds = require "lib/dtutils.string"
+
+    local result = ds.sanitize(str)
+      str - string - the string that needs to be made safe]],
+  Description = [[sanitize converts a string into a version suitable for
+    use passing as an argument in a system command.]],
+  Return_Value = [[result - string - a websafe string]],
+  Limitations = [[]],
+  Example = [[]],
+  See_Also = [[]],
+  Reference = [[]],
+  License = [[]],
+  Copyright = [[]],
+}
+
+function dtutils_string.sanitize(str)
+  local result = ""
+
+  if dt.configuration.running_os == "windows" then
+    result = '"' .. str .. '"'
+  else
+    result = "'" .. str .. "'"
+  end
+  
+  return result
+end
+
 return dtutils_string

lib/dtutils/system.lua

@@ -0,0 +1,133 @@
+local dtutils_system = {}
+
+local dt = require "darktable"
+require "official/yield"  -- necessary for dt.control.execute()
+
+dtutils_system.libdoc = {
+  Name = [[dtutils.system]],
+  Synopsis = [[a library of system utilities for use in darktable lua scripts]],
+  Usage = [[local ds = require "lib/dtutils.system"]],
+  Description = [[This library contains routines for interfacing to the operating system from
+    darktable lua scripts.]],
+  Return_Value = [[du - library - the darktable lua system library]],
+  Limitations = [[]],
+  Example = [[]],
+  See_Also = [[]],
+  Reference = [[]],
+  License = [[This program is free software: you can redistribute it and/or modify
+    it under the terms of the GNU General Public License as published by
+    the Free Software Foundation; either version 3 of the License, or
+    (at your option) any later version.
+
+    This program is distributed in the hope that it will be useful,
+    but WITHOUT ANY WARRANTY; without even the implied warranty of
+    MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+    GNU General Public License for more details.
+
+    You should have received a copy of the GNU General Public License
+    along with this program.  If not, see <http://www.gnu.org/licenses/>.]],
+  Copyright = [[Copyright (c) 2018 Bill Ferguson <wpferguson@gmail.com>]],
+  functions = {}
+}
+
+dtutils_system.libdoc.functions["external_command"] = {
+  Name = [[external_command]],
+  Synopsis = [[pass a command to the operating system for execution and return the result]],
+  Usage = [[local dsys = require "lib/dtutils.system"
+
+    local result = dsys.external_command(command)
+      command - string - a string containing the command and arguments to be passed to the operating system for execution.]],
+  Description = [[external_command passes a command to the operating system for execution and returns the results.]], 
+  Return_Value = [[result - the return value signalling success or failure.]],
+  Limitations = [[]],
+  Example = [[]],
+  See_Also = [[]],
+  Reference = [[]],
+  License = [[]],
+  Copyright = [[]],
+}
+
+function dtutils_system.external_command(command)
+  local result = nil
+
+  if dt.configuration.running_os == "windows" then
+    result = dtutils_system.windows_command(command)
+  else
+    result = dt.control.execute(command)
+  end
+
+  return result
+end
+
+dtutils_system.libdoc.functions["windows_command"] = {
+  Name = [[windows_command]],
+  Synopsis = [[pass a command to the windows operating system for execution and return the result]],
+  Usage = [[local dsys = require "lib/dtutils.system"
+
+    local result = dsys.windows_command(command)
+      command - string - a string containing the command and arguments to be passed to the operating system for execution.]],
+  Description = [[The normal method of executing an operating system command is using dt.control.execute(), but that doesn't 
+    work with Windows when more than one item in the command is quoted.  In order to ensure command execution on Windows we 
+    create a batch file in the temporary directory, put the command in it, execute the batch file, then return the result.]], 
+  Return_Value = [[result - the return value signalling success or failure.]],
+  Limitations = [[]],
+  Example = [[]],
+  See_Also = [[]],
+  Reference = [[]],
+  License = [[]],
+  Copyright = [[]],
+}
+
+function dtutils_system.windows_command(command)
+  local result = 1
+
+  local fname = dt.configuration.tmp_dir .. "/run_command.bat"
+
+  local file = io.open(fname, "w")
+  if file then
+    dt.print_log("opened file")
+    command = string.gsub(command, "%%", "%%%%") -- escape % from windows shell
+    file:write(command)
+    file:close()
+
+    result = dt.control.execute(fname)
+    dt.print_log("result from windows command was " .. result)
+
+    os.remove(fname)
+  else
+    dt.print_error("Windows command failed: unable to create batch file")
+  end
+
+  return result
+end
+
+
+dtutils_system.libdoc.functions["launch_default_app"] = {
+  Name = [[launch_default_app]],
+  Synopsis = [[open file in default application]],
+  Usage = [[local dsys = require "lib/dtutils.file"
+
+    result = dsys.launch_default_app(path)
+      path - string - a file path]],
+  Description = [[launch_default_app allows opening a file in the application that is assigned as default 
+    for that filetype in the users's system]],
+  Return_Value = [[result - the return value signalling success or failure.]],
+  Limitations = [[]],
+  Example = [[]],
+  See_Also = [[]],
+  Reference = [[]],
+  License = [[]],
+  Copyright = [[]],
+}
+function dtutils_system.launch_default_app(path) 
+  local open_cmd = "xdg-open "
+  if (dt.configuration.running_os == "windows") then
+    open_cmd = "" -- On Windows we don't need any command. (start e.g. has problems with spaces in the filename, even if we put quoter around them.) https://stackoverflow.com/questions/13691827/opening-file-with-spaces-in-windows-via-command-prompt
+  elseif  (dt.configuration.running_os == "macos") then
+    open_cmd = "open "
+  end   
+  return dtutils_system.external_command(open_cmd .. path)
+end
+
+
+return dtutils_system

locale/de_DE/LC_MESSAGES/hugin.po

@@ -8,14 +8,14 @@ msgstr ""
 "Project-Id-Version: \n"
 "Report-Msgid-Bugs-To: \n"
 "POT-Creation-Date: 2016-01-31 22:59+0100\n"
-"PO-Revision-Date: 2016-01-31 23:09+0100\n"
+"PO-Revision-Date: 2018-09-18 10:22+0200\n"
 "Language: de_DE\n"
 "MIME-Version: 1.0\n"
 "Content-Type: text/plain; charset=UTF-8\n"
 "Content-Transfer-Encoding: 8bit\n"
-"Last-Translator: Tobias Jakobs <tobias.jakobs@gmail.com>\n"
+"Last-Translator: Simon Legner <Simon.Legner@gmail.com>\n"
 "Language-Team: \n"
-"X-Generator: Poedit 1.8.4\n"
+"X-Generator: Poedit 2.1.1\n"
 
 #: hugin.lua:73
 msgid "hugin not found"
@@ -27,7 +27,7 @@ msgstr "Versuche Bilder zusammenzufügen"
 
 #: hugin.lua:102
 msgid "Creating pto file"
-msgstr "Erzeuge pto Datei"
+msgstr "Erzeuge pto-Datei"
 
 #: hugin.lua:105
 msgid "Running Assistent"
@@ -39,4 +39,4 @@ msgstr "Aufruf von hugin fehlgeschlagen ..."
 
 #: hugin.lua:124
 msgid "Hugin Panorama"
-msgstr "Hugin Panorama"
+msgstr "Hugin-Panorama"

locale/de_DE/LC_MESSAGES/kml_export.po

@@ -2,45 +2,45 @@ msgid ""
 msgstr ""
 "Project-Id-Version: Lua gettextTest\n"
 "POT-Creation-Date: 2016-01-22 15:33+0100\n"
-"PO-Revision-Date: 2016-01-22 23:28+0100\n"
-"Last-Translator: Tobias Jakobs <tobias.jakobs@gmail.com>\n"
+"PO-Revision-Date: 2018-09-18 10:21+0200\n"
+"Last-Translator: Simon Legner <Simon.Legner@gmail.com>\n"
 "Language-Team: \n"
 "Language: de\n"
 "MIME-Version: 1.0\n"
 "Content-Type: text/plain; charset=UTF-8\n"
 "Content-Transfer-Encoding: 8bit\n"
-"X-Generator: Poedit 1.8.4\n"
+"X-Generator: Poedit 2.1.1\n"
 "X-Poedit-Basepath: .\n"
-"Plural-Forms: nplurals=2;t" plural=(n != 1);\n"
+"Plural-Forms: nplurals=2;t\" plural=(n != 1);\n"
 "X-Poedit-KeywordsList: gettext;dgettext:2;dcgettext:2;ngettext:1,2;dngettext:2,3\n"
 "X-Poedit-SearchPath-0: .\n"
 
 msgid "KML/KMZ Export"
-msgstr "KML/KMZ Expor
+msgstr "KML/KMZ-Export"
 
 msgid "Export Image %i/%i"
 msgstr "Exportiere Bild %i/%i"
 
 msgid "KML export: Open KML/KMZ file after export"
-msgstr "KML export: KML/KMZ-Datei nach dem exportieren öffnen"
+msgstr "KML-Export KML/KMZ-Datei nach dem Export öffnen"
 
 msgid "Opens the KML file after the export with the standard programm for KML files"
 msgstr "Öffnet die KML-Datei mit dem Standardprogram für KML-Dateien"
 
 msgid "KML export: Export directory"
-msgstr "KML export: Export-Verzeichnis"
+msgstr "KML-Export: Export-Verzeichnis"
 
 msgid "A directory that will be used to export the KML/KMZ files"
 msgstr "Verzeichnis in das die KML/KMZ-Dateien exportiert werden"
 
 msgid "KML export: Connect images with path"
-msgstr "KML export: Bilder mit einer Linie verbinden"
+msgstr "KML-Export: Bilder mit einer Linie verbinden"
 
 msgid "connect all images with a path"
 msgstr "Verbinde alle Bilder mit einer Linie"
 
 msgid "KML export: Create KMZ file"
-msgstr "KML export: KMZ-Datei erstellen"
+msgstr "KML-Export: KMZ-Datei erstellen"
 
 msgid "Compress all imeges to one KMZ file"
 msgstr "Alle Bilder in einer KMZ-Datei komprimieren"

locale/de_DE/LC_MESSAGES/quicktag.po

@@ -0,0 +1,84 @@
+# Copyright (C) YEAR THE PACKAGE'S COPYRIGHT HOLDER
+# This file is distributed under the same license as the PACKAGE package.
+#
+# Christian Kanzian <christian.kanzian@gmx.at>, 2017.
+msgid ""
+msgstr ""
+"Project-Id-Version: \n"
+"Report-Msgid-Bugs-To: \n"
+"POT-Creation-Date: 2017-02-20 17:38+0100\n"
+"PO-Revision-Date: 2017-02-20 17:50+0100\n"
+"Last-Translator: Christian Kanzian <christian.kanzian@gmx.at>\n"
+"Language-Team: German <kde-i18n-de@kde.org>\n"
+"Language: de\n"
+"MIME-Version: 1.0\n"
+"Content-Type: text/plain; charset=UTF-8\n"
+"Content-Transfer-Encoding: 8bit\n"
+"Plural-Forms: nplurals=2; plural=(n != 1);\n"
+"X-Generator: Lokalize 1.5\n"
+
+#: quicktag.lua:64
+msgid "max. length of button labels"
+msgstr "Max. Länge der Schaltflächen-Beschriftung"
+
+#: quicktag.lua:65
+msgid "may range from 15 to 60 - needs a restart"
+msgstr "Bereich von 15 bis 60 möglich - Neustart erforderlich"
+
+#: quicktag.lua:78
+msgid "number of quicktag fields"
+msgstr "Anzahl von quicktag Feldern"
+
+#: quicktag.lua:79
+msgid "may range from 2 to 20 - needs a restart"
+msgstr "Bereich von 2 bis 20 möglich - Neustart erforderlich"
+
+#: quicktag.lua:119
+#, lua-format
+msgid "quicktag %i is empty, please set a tag"
+msgstr "quicktag %i ist leer - bitte Tag eingeben"
+
+#: quicktag.lua:134
+msgid "no images selected"
+msgstr "keine Bilder ausgewählt"
+
+#: quicktag.lua:144
+#, lua-format
+msgid "tag \"%s\" attached to %i image(s)"
+msgstr "Tag \"%s\" an  %i Bild(er) angehängt"
+
+#: quicktag.lua:165
+msgid "old quicktag"
+msgstr "Alter quicktag"
+
+#: quicktag.lua:166
+msgid "select the quicktag to replace"
+msgstr "Wähle den quicktag der ersetzt werden soll"
+
+#: quicktag.lua:182 quicktag.lua:206
+msgid "new quicktag"
+msgstr "Neuer quicktag"
+
+#: quicktag.lua:185
+msgid "enter your tag here"
+msgstr "Bitte Tag hier eingeben"
+
+#: quicktag.lua:189
+msgid "set quicktag"
+msgstr "Setze Tag"
+
+#: quicktag.lua:193
+msgid "new quicktag is empty!"
+msgstr "Neuer Tag ist leer!"
+
+#: quicktag.lua:197
+#, lua-format
+msgid "quicktag \"%s\" replaced by \"%s\""
+msgstr "quicktag \"%s\" ersetzt durch \"%s\" "
+
+#: quicktag.lua:246
+#, lua-format
+msgid "quicktag %i"
+msgstr "quicktag %i"
+
+

locale/de_DE/LC_MESSAGES/select_untagged.po

@@ -0,0 +1,27 @@
+msgid ""
+msgstr ""
+"Project-Id-Version: \n"
+"POT-Creation-Date: 2017-09-14 12:18+0200\n"
+"PO-Revision-Date: 2017-09-14 12:23+0200\n"
+"Last-Translator: Jannis_V\n"
+"Language-Team: \n"
+"Language: de_DE\n"
+"MIME-Version: 1.0\n"
+"Content-Type: text/plain; charset=UTF-8\n"
+"Content-Transfer-Encoding: 8bit\n"
+"X-Generator: Poedit 1.8.7.1\n"
+"X-Poedit-Basepath: ../../../contrib\n"
+"Plural-Forms: nplurals=2; plural=(n != 1);\n"
+"X-Poedit-SearchPath-0: select_untagged.lua\n"
+
+#: select_untagged.lua:38
+msgid "select untagged images"
+msgstr "Auswahl von Bildern ohne Tags"
+
+#: select_untagged.lua:64
+msgid "select untagged"
+msgstr "Ohne Tags"
+
+#: select_untagged.lua:64
+msgid "select all images containing no tags or only tags added by darktable"
+msgstr "Wähle alle Bilder der aktuellen Sammlung aus, die keine Tags oder nur automatisch durch darktable erzeugte Tags enthalten"

locale/fr_FR/LC_MESSAGES/pdf_slideshow.po

@@ -0,0 +1,70 @@
+# SOME DESCRIPTIVE TITLE.
+# Copyright (C) YEAR THE PACKAGE'S COPYRIGHT HOLDER
+# This file is distributed under the same license as the PACKAGE package.
+# FIRST AUTHOR <EMAIL@ADDRESS>, YEAR.
+#
+msgid ""
+msgstr ""
+"Project-Id-Version: \n"
+"Report-Msgid-Bugs-To: \n"
+"POT-Creation-Date: 2017-04-10 09:30+0200\n"
+"PO-Revision-Date: 2017-04-10 09:48+0200\n"
+"Language: fr_FR\n"
+"MIME-Version: 1.0\n"
+"Content-Type: text/plain; charset=UTF-8\n"
+"Content-Transfer-Encoding: 8bit\n"
+"Last-Translator: Pascal Obry <pascal@obry.net>\n"
+"Language-Team: \n"
+"X-Generator: Poedit 1.8.11\n"
+
+#: pdf_slideshow.lua:55
+msgid "pdflatex not found"
+msgstr "pdflatex non trouvé"
+
+#: pdf_slideshow.lua:63
+msgid "a pdf viewer"
+msgstr "lecteur pdf"
+
+#: pdf_slideshow.lua:64
+msgid "can be an absolute pathname or the tool may be in the PATH"
+msgstr "peut être un chemin absolu où l'outil peut être dans le chemin par défaut"
+
+#: pdf_slideshow.lua:68 pdf_slideshow.lua:95
+msgid "slideshow title"
+msgstr "titre du diaporama"
+
+#: pdf_slideshow.lua:72
+msgid "transition delay (s)"
+msgstr "temporisation (s) entre diapositives"
+
+#: pdf_slideshow.lua:83
+msgid "include image title"
+msgstr "inclus le titre de l'image"
+
+#: pdf_slideshow.lua:85
+msgid "whether to include the image title (if defined) into the slide"
+msgstr "indique si le titre de l'image (si défini) doit être inclus dans la diapositive"
+
+#: pdf_slideshow.lua:89
+msgid "include image author"
+msgstr "inclus l'auteur de l'image"
+
+#: pdf_slideshow.lua:91
+msgid "whether to include the image author (if defined) into the slide"
+msgstr "indique si l'auteur de l'image (si défini) doit être inclus dans la diapositive"
+
+#: pdf_slideshow.lua:167
+msgid "pdf slideshow"
+msgstr "diaporama pdf"
+
+#: pdf_slideshow.lua:220
+msgid "problem running pdflatex"
+msgstr "impossible d'exécuter pdflatex"
+
+#: pdf_slideshow.lua:221 pdf_slideshow.lua:231
+msgid "problem running "
+msgstr "impossible d'exécuter "
+
+#: pdf_slideshow.lua:230
+msgid "problem running pdf viewer"
+msgstr "impossible d'exécuter le lecteur pdf"

locale/fr_FR/LC_MESSAGES/select_untagged.po

@@ -0,0 +1,27 @@
+msgid ""
+msgstr ""
+"Project-Id-Version: \n"
+"POT-Creation-Date: 2017-09-14 15:42+0200\n"
+"PO-Revision-Date: 2017-09-14 15:52+0200\n"
+"Last-Translator: Jannis_V\n"
+"Language-Team: \n"
+"Language: fr_FR\n"
+"MIME-Version: 1.0\n"
+"Content-Type: text/plain; charset=UTF-8\n"
+"Content-Transfer-Encoding: 8bit\n"
+"X-Generator: Poedit 1.8.7.1\n"
+"X-Poedit-Basepath: ../../../contrib\n"
+"Plural-Forms: nplurals=2; plural=(n > 1);\n"
+"X-Poedit-SearchPath-0: select_untagged.lua\n"
+
+#: select_untagged.lua:38
+msgid "select untagged images"
+msgstr "sélection des images sans mots-clés"
+
+#: select_untagged.lua:64
+msgid "select untagged"
+msgstr "sans mots-clés"
+
+#: select_untagged.lua:64
+msgid "select all images containing no tags or only tags added by darktable"
+msgstr "sélection des images sans mots-clés ou seulement avec les mots-clés ajoutés par darktable"

official/check_for_updates.lua

@@ -28,10 +28,11 @@ USAGE
 ]]
 
 local dt = require "darktable"
+local du = require "lib/dtutils"
 local https = require "ssl.https"
 local cjson = require "cjson"
 
-dt.configuration.check_version(...,{2,0,0},{3,0,0})
+du.check_min_api_version("2.0.0", "check_for_updates") 
 
 -- compare two version strings of the form "major.minor.patch"
 -- returns -1, 0, 1 if the first version is smaller, equal, greater than the second version,

official/copy_paste_metadata.lua

@@ -26,8 +26,9 @@ USAGE
 ]]
 
 local dt = require "darktable"
+local du = require "lib/dtutils"
 
-dt.configuration.check_version(...,{3,0,0})
+du.check_min_api_version("3.0.0", "copy_paste_metadata")
 
 -- set this to "false" if you don't want to overwrite metadata fields
 -- (title, description, creator, publisher and rights) that are already set

official/delete_long_tags.lua

@@ -1,6 +1,6 @@
 --[[
     This file is part of darktable,
-    copyright (c) 2014 Tobias Ellinghaus
+    copyright (c) 2014--2018 Tobias Ellinghaus
 
     darktable is free software: you can redistribute it and/or modify
     it under the terms of the GNU General Public License as published by
@@ -17,7 +17,7 @@
 ]]
 --[[
 DELETE LONG TAGS
-A simple script that will automatically delete all tag longer than a set length
+A simple script that will automatically delete all tags longer than a set length
 
 USAGE
 * require this script from your main lua file
@@ -29,7 +29,9 @@ all tags longer than the given length will be automatically deleted at every res
 ]]
 
 local dt = require "darktable"
-dt.configuration.check_version(...,{2,0,0},{3,0,0})
+local du = require "lib/dtutils"
+
+du.check_min_api_version("2.0.0", "delete_long_tags") 
 
 dt.preferences.register("delete_long_tags", "length", "integer",
                         "maximum length of tags to keep",
@@ -38,10 +40,18 @@ dt.preferences.register("delete_long_tags", "length", "integer",
 
 local max_length = dt.preferences.read("delete_long_tags", "length", "integer")
 
+-- deleting while iterating the tags list seems to break the iterator!
+local long_tags = {}
+
 for _,t in ipairs(dt.tags) do
   local len = #t.name
   if len > max_length then
     print("deleting tag `"..t.name.."' (length: "..len..")")
-    t:delete()
+    table.insert(long_tags, t.name)
   end
 end
+
+for _,name in pairs(long_tags) do
+  tag = dt.tags.find(name)
+  tag:delete()
+end

official/delete_unused_tags.lua

@@ -0,0 +1,47 @@
+--[[
+    This file is part of darktable,
+    copyright (c) 2018 Tobias Ellinghaus
+
+    darktable is free software: you can redistribute it and/or modify
+    it under the terms of the GNU General Public License as published by
+    the Free Software Foundation, either version 3 of the License, or
+    (at your option) any later version.
+
+    darktable is distributed in the hope that it will be useful,
+    but WITHOUT ANY WARRANTY; without even the implied warranty of
+    MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+    GNU General Public License for more details.
+
+    You should have received a copy of the GNU General Public License
+    along with darktable.  If not, see <http://www.gnu.org/licenses/>.
+]]
+--[[
+DELETE UNUSED TAGS
+A simple script that will automatically delete all tags that are not attached to any images
+
+USAGE
+* require this script from your main lua file
+* restart darktable
+
+all tags that are not used will be automatically deleted at every restart
+]]
+
+local dt = require "darktable"
+local du = require "lib/dtutils"
+
+du.check_min_api_version("5.0.0", "delete_unused_tags") 
+
+-- deleting while iterating the tags list seems to break the iterator!
+local unused_tags = {}
+
+for _, t in ipairs(dt.tags) do
+  if #t == 0 then
+    table.insert(unused_tags, t.name)
+  end
+end
+
+for _,name in pairs(unused_tags) do
+  print("deleting tag `" .. name .. "'")
+  tag = dt.tags.find(name)
+  tag:delete()
+end

official/enfuse.lua

@@ -17,7 +17,7 @@
 ]]
 --[[
 RUN ENFUSE ON THE SELECTED IMAGES
-This script uses enfuse to merge the selected images into one HDR and imports the result.
+This script uses enfuse to merge the selected images into one tonemapped image and imports the result.
 It only works on ldr images (like, JPEG).
 
 USAGE
@@ -33,46 +33,77 @@ TODO
 ]]
 
 local dt = require "darktable"
-require "official/yield"
+local du = require "lib/dtutils"
+local df = require "lib/dtutils.file"
+local dtsys = require "lib/dtutils.system"
 
+local PS = dt.configuration.running_os == "windows" and "\\" or "/"
 
-dt.configuration.check_version(...,{3,0,0})
+local gettext = dt.gettext
 
--- thanks Tobias Jakobs for this function (taken from contrib/hugin.lua)
-local function checkIfBinExists(bin)
-  local handle = io.popen("which "..bin)
-  local result = handle:read()
-  local ret
-  handle:close()
-  if (result) then
-    dt.print_error("true checkIfBinExists: "..bin)
-    ret = true
-  else
-    dt.print_error(bin.." not found")
-    ret = false
-  end
-  return ret
+du.check_min_api_version("3.0.0", "enfuse")
+
+-- Tell gettext where to find the .mo file translating messages for a particular domain
+gettext.bindtextdomain("enfuse",dt.configuration.config_dir..PS .. "lua" .. PS .. "locale" .. PS)
+
+local function _(msgid)
+    return gettext.dgettext("enfuse", msgid)
 end
 
 -- add a new lib
 -- is enfuse installed?
-local enfuse_installed = checkIfBinExists("enfuse")
+local enfuse_installed = df.check_if_bin_exists("enfuse")
+
+-- instance of DT tiff exporter
+local tiff_exporter = dt.new_format("tiff")
+tiff_exporter.bpp = 16
+tiff_exporter.max_height = 0
+tiff_exporter.max_width = 0
+
+-- check the version so that we can use the correct arguments
+
+local version = nil
+
+local p = io.popen(enfuse_installed .. " --version")
+local f = p:read("all")
+p:close()
+version = string.match(f, "enfuse (%d.%d)")
+dt.print_log("enfuse version is " .. version)
+
 
 -- initialize exposure_mu value and depth setting in config to sane defaults (would be 0 otherwise)
 if dt.preferences.read("enfuse", "depth", "integer") == 0 then
   dt.preferences.write("enfuse", "depth", "integer", 2)
   dt.preferences.write("enfuse", "exposure_mu", "float", 0.5)
+  dt.preferences.write("enfuse", "exposure_optimum", "float", 0.5)
 end
 
 -- set up some widgets, initialized from config
-local exposure_mu = dt.new_widget("slider")
-{
-  label = "exposure mu",
-  tooltip = "center also known as MEAN of Gaussian weighting function (0 <= MEAN <= 1); default: 0.5",
-  hard_min = 0,
-  hard_max = 1,
-  value = dt.preferences.read("enfuse", "exposure_mu", "float")
-}
+
+local exposure_mu = nil
+
+if version < "4.2" then
+  exposure_mu = dt.new_widget("slider")
+  {
+    label = "exposure mu",
+    tooltip = "center also known as MEAN of Gaussian weighting function (0 <= MEAN <= 1); default: 0.5",
+    hard_min = 0,
+    hard_max = 1,
+    value = dt.preferences.read("enfuse", "exposure_mu", "float")
+  }
+else
+  exposure_mu = dt.new_widget("slider")
+  {
+    label = "exposure optimum",
+    tooltip = "optimum exposure value, usually the maximum of the weighting function (0 <= OPTIMUM <=1); default 0.5",
+    hard_min = 0,
+    hard_max = 1,
+    value = dt.preferences.read("enfuse", "exposure_optimum", "float")
+  }
+end
+  
+
+
 
 local depth = dt.new_widget("combobox")
 {
@@ -83,6 +114,115 @@ local depth = dt.new_widget("combobox")
   "8", "16", "32"
 }
 
+local enfuse_button = dt.new_widget("button")
+{
+  label = enfuse_installed and "run enfuse" or "enfuse not installed",
+  clicked_callback = function ()
+    -- remember exposure_mu
+    -- TODO: find a way to save it whenever the value changes
+    local mu = exposure_mu.value
+    if version < "4.2" then
+      dt.preferences.write("enfuse", "exposure-mu", "float", mu)
+    else
+      dt.preferences.write("enfuse", "exposure-optimum", "float", mu)
+    end
+
+    -- create a temp response file
+    local response_file = os.tmpname()
+    if dt.configuration.running_os == "windows" then
+      response_file = dt.configuration.tmp_dir .. response_file -- windows os.tmpname() defaults to root directory
+    end
+    local f = io.open(response_file, "w")
+    if not f then
+      dt.print(string.format(_("Error writing to `%s`"), response_file))
+      os.remove(response_file)
+      return
+    end
+
+    -- add all filenames to the response file
+    local cnt = 0
+    local n_skipped = 0
+    local target_dir
+    for i_, i in ipairs(dt.gui.action_images) do
+
+      -- only use ldr files as enfuse can't open raws
+      if i.is_ldr then
+        cnt = cnt + 1
+        f:write(i.path..PS..i.filename.."\n")
+        target_dir = i.path
+
+      -- alternatively raws will be exported as tiff
+      elseif i.is_raw then
+        local tmp_exported = os.tmpname()..".tif"
+        if dt.configuration.running_os == "windows" then
+          tmp_exported = dt.configuration.tmp_dir .. tmp_exported -- windows os.tmpname() defaults to root directory
+        end
+            dt.print(string.format(_("Converting raw file '%s' to tiff..."), i.filename)) 
+        tiff_exporter:write_image(i, tmp_exported, false)
+        dt.print_log(string.format("Raw file '%s' converted to '%s'", i.filename, tmp_exported))
+
+        cnt = cnt + 1
+        f:write(tmp_exported.."\n")
+        target_dir = i.path
+        
+      -- other images will be skipped
+      else
+        dt.print(string.format(_("Skipping %s..."), i.filename))
+        n_skipped = n_skipped + 1
+      end
+    end
+    f:close()
+    -- bail out if there is nothing to do
+    if cnt == 0 then
+      dt.print(_("No suitable images selected, nothing to do for enfuse"))
+      os.remove(response_file)
+      return
+    end
+
+    if n_skipped > 0 then
+      dt.print(string.format(_("%d image(s) skipped"), n_skipped))
+    end
+
+    -- call enfuse on the response file
+    -- TODO: find something nicer
+    local ugly_decimal_point_hack = string.gsub(string.format("%.04f", mu), ",", ".")
+    -- TODO: make filename unique
+    local output_image = target_dir.. PS .. "enfuse.tif"
+    local exposure_option = " --exposure-optimum "
+    if version < "4.2" then
+      exposure_option = " --exposure-mu "
+    end
+    local command = enfuse_installed.." --depth "..depth.value..exposure_option..ugly_decimal_point_hack
+                    .." -o \""..output_image.."\" \"@"..response_file.."\""
+    if dtsys.external_command( command) > 0 then
+      dt.print(_("Enfuse failed, see terminal output for details"))
+      os.remove(response_file)
+      return
+    end
+
+    -- remove the response file
+    os.remove(response_file)
+
+    -- import resulting tiff
+    local image = dt.database.import(output_image)
+
+    -- tell the user that everything worked
+    dt.print(_("enfuse was successful, resulting image has been imported"))
+    -- normally printing to stdout is bad, but we allow enfuse to show its output, so adding one extra line is ok
+    print(string.format(_("enfuse: done, resulting image '%s' has been imported with id %d"), output_image, image.id))
+  end
+}
+
+local lib_widgets = {}
+
+if not enfuse_installed then
+  table.insert(lib_widgets, df.executable_path_widget({"ffmpeg"}))
+end
+table.insert(lib_widgets, exposure_mu)
+table.insert(lib_widgets, depth)
+table.insert(lib_widgets, enfuse_button)
+
+ 
 -- ... and tell dt about it all
 dt.register_lib(
   "enfuse",                                                                    -- plugin name
@@ -94,79 +234,7 @@ dt.register_lib(
   {
     orientation = "vertical",
     sensitive = enfuse_installed,
-    exposure_mu,
-    depth,
-    dt.new_widget("button")
-    {
-      label = enfuse_installed and "run enfuse" or "enfuse not installed",
-      clicked_callback = function (_)
-        -- remember exposure_mu
-        -- TODO: find a way to save it whenever the value changes
-        local mu = exposure_mu.value
-        dt.preferences.write("enfuse", "exposure_mu", "float", mu)
-
-        -- create a temp response file
-        local response_file = os.tmpname()
-        local f = io.open(response_file, "w")
-        if not f then
-          dt.print("error writing to `"..response_file.."'")
-          os.remove(response_file)
-          return
-        end
-
-        -- add all filenames to the response file
-        local cnt = 0
-        local n_skipped = 0
-        local target_dir
-        for _, i in ipairs(dt.gui.action_images) do
-          -- only use ldr files as enfuse can't open raws. alternatively we could export raws that we encounter
-          if i.is_ldr then
-            cnt = cnt + 1
-            f:write(i.path.."/"..i.filename.."\n")
-            target_dir = i.path
-          else
-            dt.print("skipping "..i.filename)
-            n_skipped = n_skipped + 1
-          end
-        end
-        f:close()
-        -- bail out if there is nothing to do
-        if cnt == 0 then
-          dt.print("no suitable images selected, nothing to do for enfuse")
-          os.remove(response_file)
-          return
-        end
-        if n_skipped == 1 then
-          dt.print(n_skipped.." image skipped")
-        elseif n_skipped > 1 then
-          dt.print(n_skipped.." images skipped")
-        end
-
-        -- call enfuse on the response file
-        -- TODO: find something nicer
-        local ugly_decimal_point_hack = string.gsub(string.format("%.04f", mu), ",", ".")
-        -- TODO: make filename unique
-        local output_image = target_dir.."/enfuse.tiff"
-        local command = "enfuse --depth "..depth.value.." --exposure-mu "..ugly_decimal_point_hack
-                        .." -o \""..output_image.."\" \"@"..response_file.."\""
-        if dt.control.execute( command) > 0 then
-          dt.print("enfuse failed, see terminal output for details")
-          os.remove(response_file)
-          return
-        end
-
-        -- remove the response file
-        os.remove(response_file)
-
-        -- import resulting tiff
-        local image = dt.database.import(output_image)
-
-        -- tell the user that everything worked
-        dt.print("enfuse was successful, resulting image was imported")
-        -- normally printing to stdout is bad, but we allow enfuse to show its output, so adding one extra line is ok
-        print("enfuse: done, resulting image '"..output_image.."' was imported with id "..image.id)
-      end
-    }
+    table.unpack(lib_widgets)
   },
   nil,-- view_enter
   nil -- view_leave

official/generate_image_txt.lua

@@ -35,10 +35,11 @@ USAGE
 --  * make filenames with double quotes (") work
 
 local dt = require "darktable"
+local du = require "lib/dtutils"
 require "darktable.debug"
 require "official/yield"
 
-dt.configuration.check_version(...,{2,1,0},{3,0,0})
+du.check_min_api_version("2.1.0", "generate_image_txt") 
 
 dt.preferences.register("generate_image_txt",
                         "enabled",

official/image_path_in_ui.lua

@@ -29,7 +29,9 @@ This plugin will add a widget at the bottom of the left column in lighttable mod
 
 ]]
 local dt = require "darktable"
-dt.configuration.check_version(...,{2,0,0},{3,0,0})
+local du = require "lib/dtutils"
+
+du.check_min_api_version("2.0.0", "image_path_in_ui") 
 
 local main_label = dt.new_widget("label"){selectable = true, ellipsize = "middle", halign = "start"}
 

official/import_filter_manager.lua

@@ -67,7 +67,7 @@ end)
 -- add a new global function to register import filters
 dt.register_import_filter = function(name, callback)
   local active_filter = dt.preferences.read("import_filter_manager", "active_filter", "string")
-  dt.print_error("registering import filter `" .. name .. "'")
+  dt.print_log("registering import filter `" .. name .. "'")
   import_filter_list[name] = callback
   n_import_filters = n_import_filters  + 1
   filter_dropdown[n_import_filters] = name

official/save_selection.lua

@@ -34,7 +34,9 @@ increase it if you need more temporary selection buffers
 
 ]]
 local dt = require "darktable"
-dt.configuration.check_version(...,{2,0,0},{3,0,0})
+local du = require "lib/dtutils"
+
+du.check_min_api_version("2.0.0", "save_selection") 
 
 local buffer_count = 5
 

official/selection_to_pdf.lua

@@ -34,9 +34,10 @@ Plugin allows you to choose how many thumbnails you need per row
 
 ]]
 local dt = require "darktable"
+local du = require "lib/dtutils"
 require "official/yield"
 
-dt.configuration.check_version(...,{2,0,0},{3,0,0})
+du.check_min_api_version("2.0.0", "selection_to_pdf")
 
 dt.preferences.register
    ("selection_to_pdf","Open with","string",

official/yield.lua

@@ -25,21 +25,6 @@ USAGE
 local dt = require "darktable"
 local yield_orig = coroutine.yield
 
--- ToDo: Test with dt master
---[[
-if (dt.configuration.api_version_major >= 4) then
-  coroutine.yield = function(yield_type, command)
-    if (yield_type == "RUN_COMMAND") then
-      dt.control.execute(command)
-    elseif (yield_type == "FILE_READABLE") then
-      dt.control.read(command)
-    elseif (yield_type == "WAIT_MS") then
-      dt.control.sleep(command)
-    end  
-  end
-end
-]]
-
 if (dt.configuration.api_version_major < 4) then
   dt.control = {}
   dt.control.execute = function(command)

tools/executable_manager.lua

@@ -0,0 +1,192 @@
+--[[
+  This file is part of darktable,
+  copyright (c) 2018 Bill Ferguson
+  
+  darktable is free software: you can redistribute it and/or modify
+  it under the terms of the GNU General Public License as published by
+  the Free Software Foundation, either version 3 of the License, or
+  (at your option) any later version.
+  
+  darktable is distributed in the hope that it will be useful,
+  but WITHOUT ANY WARRANTY; without even the implied warranty of
+  MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+  GNU General Public License for more details.
+  
+  You should have received a copy of the GNU General Public License
+  along with darktable.  If not, see <http://www.gnu.org/licenses/>.
+]]
+--[[
+    executable_manager.lua - a tool for managing external executables used by darktable lua scripts
+
+    executable_manager is a tool for managing the executable preferences stored in the darktablerc file.
+    On startup the darktablerc file is scanned and a widget is built for each executable path.  The user
+    can select the executable from a drop down list and then modify the settings as desired.
+
+    Any changes made using executable_manager won't be saved in the darktablerc file until darktable exits, but
+    the preference is updated when the change is made so scripts will pick up the changes without restarting
+    darktable.
+
+]]
+
+local dt = require "darktable"
+local du = require "lib/dtutils"
+local df = require "lib/dtutils.file"
+
+du.check_min_api_version("5.0.0", "executable_manager")
+
+local PS = dt.configuration.running_os == "windows" and "\\" or "/"
+
+local gettext = dt.gettext
+
+gettext.bindtextdomain("executable_manager",dt.configuration.config_dir.."/lua/locale/")
+
+-- - - - - - - - - - - - - - - - - - - - - - - - - - - - 
+-- F U N C T I O N S
+-- - - - - - - - - - - - - - - - - - - - - - - - - - - - 
+
+local function _(msg)
+    return gettext.dgettext("executable_manager", msg)
+end
+
+local function grep(file, pattern)
+
+  local result = {}
+
+  if dt.configuration.running_os == "windows" then
+    -- use find to get the matches
+    local command = "\\windows\\system32\\find.exe " .. "\"" .. pattern .. "\"" .. " " .. file
+    local f = io.popen(command)
+    local output = f:read("all")
+    f:close()
+    -- strip out the first line
+    result = du.split(output, "\n")
+    table.remove(result, 1)
+  else
+    -- use grep and just return the answers
+    local command = "grep " .. pattern .. " " .. file
+    local f = io.popen(command)
+    local output = f:read("all")
+    f:close()
+    result = du.split(output, "\n")
+  end
+  return result
+end
+
+local function update_combobox_choices(combobox, choice_table, selected)
+  local items = #combobox
+  local choices = #choice_table
+  for i, name in ipairs(choice_table) do 
+    combobox[i] = name
+  end
+  if choices < items then
+    for j = items, choices + 1, -1 do
+      combobox[j] = nil
+    end
+  end
+  combobox.value = selected
+end
+
+-- - - - - - - - - - - - - - - - - - - - - - - - - - - - 
+-- M A I N   P R O G R A M
+-- - - - - - - - - - - - - - - - - - - - - - - - - - - - 
+
+local exec_man = {} -- our own namespace
+
+local DARKTABLERC = dt.configuration.config_dir .. PS .. "darktablerc"
+
+
+-- grep the executable_paths statements out of the darktablerc file
+
+local matches = grep(DARKTABLERC, "executable_paths")
+
+-- check if we have something to manage and exit if not
+
+if #matches == 0 then
+  dt.print(_("No executable paths found, exiting..."))
+  return
+end
+
+-- build a table of the path preferences
+
+local exec_table = {}
+
+for _,pref  in ipairs(matches) do
+  local parts = du.split(pref, "=")
+  local tmp = du.split(parts[1], "/") -- preferences are stored with forward slashes
+  table.insert(exec_table, tmp[#tmp])
+end
+
+local executable_path_widgets = {}
+
+for i,exec in ipairs(exec_table) do 
+  executable_path_widgets[exec] = dt.new_widget("file_chooser_button"){
+    title = _("select ") .. exec .. _(" executable"),
+    value = df.get_executable_path_preference(exec),
+    is_directory = false,
+    changed_callback = function(self)
+      if df.check_if_bin_exists(self.value) then
+        df.set_executable_path_preference(exec, self.value)
+      end
+    end
+  }
+end
+
+-- create a stack widget to hold the executable path widgets
+
+exec_man.stack = dt.new_widget("stack"){}
+
+
+-- create a combobox to for indexing into the stack of widgets
+
+exec_man.selector = dt.new_widget("combobox"){
+  label = "executable",
+  tooltip = _("select executable to modify"),
+  value = 1, "placeholder",
+  changed_callback = function(self)
+    for i,exec in ipairs(exec_table) do
+      if self.value == exec then
+        exec_man.stack.active = i
+      end
+    end
+  end
+}
+
+-- loop through the tabke of path preferences and populate the widgets
+
+for i,exec in ipairs(exec_table) do
+  exec_man.stack[i] = dt.new_widget("box"){
+    executable_path_widgets[exec],
+    dt.new_widget("button"){
+      label = "clear",
+      tooltip = _("Clear path for ") .. exec,
+      clicked_callback = function()
+        df.set_executable_path_preference(exec, "")
+        executable_path_widgets[exec].value = ""
+      end
+    }
+
+  }
+  df.executable_path_widget({exec})
+end
+
+update_combobox_choices(exec_man.selector, exec_table, 1)
+
+-- register the lib
+
+
+dt.register_lib(
+  "executable_manager",     -- Module name
+  "executable manager",     -- Visible name
+  true,                -- expandable
+  false,               -- resetable
+  {[dt.gui.views.lighttable] = {"DT_UI_CONTAINER_PANEL_LEFT_BOTTOM", 100}},   -- containers
+  dt.new_widget("box") -- widget
+  {
+    orientation = "vertical",
+    exec_man.selector,
+    exec_man.stack,
+  },
+  nil,-- view_enter
+  nil -- view_leave
+)
+

tools/gen_i18n_mo.lua

@@ -0,0 +1,99 @@
+--[[
+    gen_18n_mo.lua - generate .mo files from .po files and put them in the correct place
+
+    Copyright (C) 2016,2018 Bill Ferguson <wpferguson@gmail.com>
+
+    This program is free software: you can redistribute it and/or modify
+    it under the terms of the GNU General Public License as published by
+    the Free Software Foundation; either version 3 of the License, or
+    (at your option) any later version.
+
+    This program is distributed in the hope that it will be useful,
+    but WITHOUT ANY WARRANTY; without even the implied warranty of
+    MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+    GNU General Public License for more details.
+
+    You should have received a copy of the GNU General Public License
+    along with this program.  If not, see <http://www.gnu.org/licenses/>.
+]]
+--[[
+    gen_i18n_mo - generate translation files from the source and place them in the appropriate locale directory
+    
+    gen_i18n_mo finds all the .po files scattered throughout the script tree, compiles them into
+    .mo files and places them in the correct locale directory for use by the gettext tools.
+
+]]
+
+local dt = require "darktable"
+local du = require "lib/dtutils"
+local df = require "lib/dtutils.file"
+local log = require "lib/dtutils.log"
+local dtsys = require "lib/dtutils.system"
+
+du.check_min_api_version("5.0.0", "gen_I18n_mo")
+
+-- figure out the path separator
+
+local PS = dt.configuration.running_os == "windows" and "\\" or "/"
+
+local LUA_DIR = dt.configuration.config_dir .. PS .. "lua" .. PS
+local LOCALE_DIR = dt.configuration.config_dir .. PS .. "lua" .. PS .. "locale" .. PS
+
+-- check if we have msgfmt
+
+local msgfmt_executable = df.check_if_bin_exists("msgfmt")
+
+if msgfmt_executable then
+
+  -- find the .po files
+
+  local find_cmd = "find -L " .. LUA_DIR .. " -name \\*.po -print"
+  if dt.configuration.running_os == "windows" then
+    find_cmd = "dir /b/s " .. LUA_DIR .. "\\*.po"
+  end
+
+  local output = io.popen(find_cmd)
+
+  -- for each .po file....
+
+  for line in output:lines() do
+    local fname = df.get_filename(line)
+
+    -- get the language used...  this depends on the file being named using
+    -- the convention /..../lang/LC_MESSAGES/file.po where lang is de_DE, fr_FR, etc.
+
+    local path_parts = du.split(line, PS)
+    local lang  = path_parts[#path_parts - 2]
+
+    -- ensure there is a destination directory for them
+
+    local mkdir_cmd = "mkdir -p "
+    if dt.configuration.running_os == "windows" then
+      mkdir_cmd = "mkdir "
+    end
+
+    if not df.check_if_file_exists(LOCALE_DIR .. lang .. PS .. "LC_MESSAGES") then
+      log.msg(log.info, "Creating locale", lang)
+      os.execute(mkdir_cmd .. LOCALE_DIR .. lang .. PS .. "LC_MESSAGES")
+    end
+
+    -- generate the mo file
+
+    fname = string.gsub(fname, ".po$", ".mo")
+    log.msg(log.info, "Compiling translation to", fname)
+    local result = os.execute(msgfmt_executable .. " -o " .. LOCALE_DIR .. lang .. PS .. "LC_MESSAGES" .. PS .. fname .. " " .. line)
+  end
+else
+  log.msg(log.screen, "ERROR: msgfmt executable not found.  Please install or specifiy location in preferences.")
+end
+dt.preferences.register("executable_paths", "msgfmt",  -- name
+  "file", -- type
+  'gen_i18n_mo: msgfmt location', -- label
+  'Install location of msgfmt. Requires restart to take effect.',  -- tooltip
+  "msgfmt",  -- default
+  dt.new_widget("file_chooser_button"){
+    title = "Select msgfmt[.exe] file",
+    value = "",
+    is_directory = false,
+  }
+)

tools/get_lib_manpages.lua

@@ -10,7 +10,7 @@ local df = require "lib/dtutils.file"
 local log = require "lib/dtutils.log"
 local libname = nil
 
-dt.configuration.check_version(...,{3,0,0})
+du.check_min_api_version("3.0.0", "get_lib_manpages") 
 
 local keys = {"Name", "Synopsis", "Usage", "Description", "Return_Value", "Limitations", 
               "Example", "See_Also", "Reference", "License", "Copyright"}

tools/get_libdoc.lua

@@ -5,8 +5,9 @@
 ]]
 
 local dt = require "darktable"
+local du = require "lib/dtutils"
 
-dt.configuration.check_version(...,{3,0,0})
+du.check_min_api_version("3.0.0", "get_libdoc") 
 
 local keys = {"Name", "Synopsis", "Usage", "Description", "Return_Value", "Limitations", 
               "Example", "See_Also", "Reference", "License", "Copyright"}

tools/plugin_manager.lua

@@ -0,0 +1 @@
+/home/bill/src/plugins/lua-scripts/tools/plugin_manager.lua

tools/script_manager.lua

@@ -0,0 +1,774 @@
+--[[
+  This file is part of darktable,
+  copyright (c) 2018 Bill Ferguson <wpferguson@gmail.com>
+  
+  darktable is free software: you can redistribute it and/or modify
+  it under the terms of the GNU General Public License as published by
+  the Free Software Foundation, either version 3 of the License, or
+  (at your option) any later version.
+  
+  darktable is distributed in the hope that it will be useful,
+  but WITHOUT ANY WARRANTY; without even the implied warranty of
+  MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+  GNU General Public License for more details.
+  
+  You should have received a copy of the GNU General Public License
+  along with darktable.  If not, see <http://www.gnu.org/licenses/>.
+]]
+--[[
+    script_manager.lua - a tool for managing the darktable lua scripts
+
+    script_manager is designed to run as a standalone script so that it
+    may be used as a drop in luarc file in the user's $HOME/.config/darktable
+    ($HOME/AppData/Local/darktable on windows)  directory.  It may also be 
+    required from a luarc file.
+
+    On startup script_manager checks to see if there is an  existing scripts directory.  
+    If there is an existing lua scripts directory then it is read to see what scripts are present.  
+    Scripts are sorted by "category" based on what subdirectory they are found in, thus with a lua 
+    scripts directory that matched the current repository the categories would be contrib, examples, 
+    offical, and tools.  Each script has an Enable/Disable button to enable or disable the script.
+
+    A link is created to the user's Downloads directory on linux, unix and MacOS.  Windows users must create the 
+    link manually using mklink.exe.  Additional "un-official" scripts may be downloaded 
+    from other sources and placed in the users Downloads directory.  These scripts all fall in a downloads category.  
+    They also each have an Enable/Disable button.
+
+]]
+
+local dt = require "darktable"
+local du = require "lib/dtutils"
+local df = require "lib/dtutils.file"
+local dtsys = require "lib/dtutils.system"
+
+local gettext = dt.gettext
+
+
+-- Tell gettext where to find the .mo file translating messages for a particular domain
+gettext.bindtextdomain("script_manager",dt.configuration.config_dir.."/lua/locale/")
+
+local function _(msgid)
+    return gettext.dgettext("script_manager", msgid)
+end
+
+collectgarbage("stop")
+
+-- - - - - - - - - - - - - - - - - - - - - - - - 
+-- C O N S T A N T S
+-- - - - - - - - - - - - - - - - - - - - - - - - 
+-- path separator
+local PS = dt.configuration.running_os == "windows" and "\\" or "/"
+
+-- command separator
+local CS = dt.configuration.running_os == "windows" and "&" or ";"
+
+local LUA_DIR = dt.configuration.config_dir .. PS .. "lua"
+local LUA_SCRIPT_REPO = "https://github.com/darktable-org/lua-scripts.git"
+
+dt.print_log("LUA_DIR is " .. LUA_DIR)
+
+-- - - - - - - - - - - - - - - - - - - - - - - - 
+-- N A M E  S P A C E
+-- - - - - - - - - - - - - - - - - - - - - - - - 
+
+local script_manager = {}
+local sm = script_manager
+
+-- - - - - - - - - - - - - - - - - - - - - - - - 
+-- F U N C T I O N S
+-- - - - - - - - - - - - - - - - - - - - - - - - 
+
+local function prequire(script)
+  dt.print_log("Loading " .. script)
+  local status, lib = pcall(require, script)
+  if status then
+    dt.print_log("Loaded " .. script)
+  else
+    dt.print_error("Error loading " .. script)
+    dt.print_error(lib)
+  end
+  return status, lib
+end
+
+local function update_combobox_choices(combobox, choice_table, selected)
+  local items = #combobox
+  local choices = #choice_table
+  for i, name in ipairs(choice_table) do 
+    combobox[i] = name
+  end
+  if choices < items then
+    for j = items, choices + 1, -1 do
+      combobox[j] = nil
+    end
+  end
+  combobox.value = selected
+end
+
+local function install_scripts()
+  local result = false
+
+  if df.check_if_file_exists(LUA_DIR) then
+    if df.check_if_file_exists(LUA_DIR .. ".orig") then
+      if dt.configuration.running_os == "windows" then
+        os.execute("rmdir /s " .. LUA_DIR .. ".orig")
+      else
+        os.execute("rm -rf " .. LUA_DIR .. ".orig")
+      end
+    end
+    os.rename(LUA_DIR, LUA_DIR .. ".orig")
+  end
+
+  local git = df.check_if_bin_exists("git")
+
+  if not git then
+    dt.print("ERROR: git not found.  Install or specify the location of the git executable.")
+    return
+  end
+
+  local git_command = git .. " clone " .. sm.lua_repository .. " " .. LUA_DIR
+  dt.print_log("install git command is " .. git_command)
+
+  result = os.execute(git_command)
+
+  if not df.check_if_file_exists(LUA_DIR .. PS .. "downloads") then
+    os.execute("mkdir " .. LUA_DIR .. PS .. "downloads")
+  end
+
+  return result
+end
+
+local function update_scripts()
+  local result = false
+
+  local git = df.check_if_bin_exists("git")
+
+  if not git then
+    dt.print("ERROR: git not found.  Install or specify the location of the git executable.")
+    return
+  end
+
+  local git_command = "cd " .. LUA_DIR .. " " .. CS .. " " .. git .. " pull"
+  dt.print_log("update git command is " .. git_command)
+
+  if dt.configuration.running_os == "windows" then
+    result = dtsys.windows_command(git_command)
+  else
+    result = os.execute(git_command)
+  end
+
+  if dt.preferences.read("script_manager", "use_lua_scripts_version", "bool") and 
+     dt.configuration.running_os == "windows" then
+    use_lua_scripts_version()
+  end
+
+  return result
+end
+
+local function add_script_data(script_file)
+
+  -- the script file supplied is category/filename.filetype
+  -- the following pattern splits the string into category, path, name, fileename, and filetype
+  -- for example contrib/gimp.lua becomes
+  -- category - contrib
+  -- path - 
+  -- name - gimp.lua
+  -- filename - gimp
+  -- filetype - lua
+
+  -- Thanks Tobias Jakobs for the awesome regulary expression
+
+  local pattern = "(.-)/(.-)(([^\\/]-)%.?([^%.\\/]*))$"
+  if dt.configuration.running_os == "windows" then
+    -- change the path separator from / to \ for windows
+    pattern = "(.-)\\(.-)(([^\\]-)%.?([^%.\\]*))$"
+  end
+
+  dt.print_log("processing " .. script_file)
+
+  -- add the script data
+  local category,path,name,filename,filetype = string.match(script_file, pattern)
+  dt.print_log("category is " .. category)
+  dt.print_log("name is " .. name)
+
+  if #sm.script_categories == 0 or not string.match(du.join(sm.script_categories, " "), category) then
+    sm.script_categories[#sm.script_categories + 1] = category
+    sm.script_names[category] = {}
+  end
+  if name then
+    dt.print_log("category is " .. category)
+    dt.print_log("name is " .. name)
+    if not string.match(du.join(sm.script_names[category], " "), name) then
+      sm.script_names[category][#sm.script_names[category] + 1] = name
+      sm.script_paths[category .. "/" .. name] = category .. "/" .. path .. name
+      if category == "downloads" then
+        sm.have_downloads = true
+      end
+    end
+  end
+end
+
+local function scan_scripts()
+  local find_cmd = "find -L " .. LUA_DIR .. " -name \\*.lua -print | sort"
+  if dt.configuration.running_os == "windows" then
+    find_cmd = "dir /b/s " .. LUA_DIR .. "\\*.lua | sort"
+  end
+  -- scan the scripts
+  local output = io.popen(find_cmd)
+  for line in output:lines() do
+    local l = string.gsub(line, LUA_DIR .. PS, "") -- strip the lua dir off
+    local script_file = l:sub(1,-5)
+    if not string.match(script_file, "script_manager") then  -- let's not include ourself
+      if not string.match(script_file, "plugins") then         -- skip plugins
+        if not string.match(script_file, "lib" .. PS) then       -- let's not try and run libraries
+          if not string.match(script_file, "include_all") then     -- skip include_all.lua
+            if not string.match(script_file, "yield") then          -- special case, because everything needs this
+              add_script_data(script_file)
+            else
+              prequire(script_file) -- load yield.lua
+            end
+          end
+        end
+      end
+    end
+  end
+  -- work around because we can't dynamically add a new stack child.  We create an empty child that will be
+  -- populated with downloads as they occur.  If there are already downloads then this is just ignored
+
+  add_script_data("downloads" .. PS)
+end
+
+-- get the script documentation, with some assumptions
+local function get_script_doc(script)
+  local description = nil
+  f = io.open(LUA_DIR .. PS .. script .. ".lua")
+  if f then
+    -- slurp the file
+    local content = f:read("*all")
+    f:close()
+    -- assume that the second block comment is the documentation
+    description = string.match(content, "%-%-%[%[.-%]%].-%-%-%[%[(.-)%]%]")
+  else
+    dt.print_error("Cant read from " .. script)
+  end
+  if description then
+    return description
+  else
+    return "No documentation available"
+  end
+end
+
+local function activate(script, scriptname)
+  dt.print_log("activating " .. scriptname)
+  local status, err = prequire(sm.script_paths[script])
+  if status then
+    dt.preferences.write("script_manager", script, "bool", true)
+    dt.print("Loaded " .. scriptname)
+  else
+    dt.print(scriptname .. " failed to load")
+    dt.print_error("Error loading " .. scriptname)
+    dt.print_error("Error message: " .. err)
+  end
+  return status
+end
+
+local function deactivate(script, scriptname)
+  -- presently the lua api doesn't support unloading gui elements therefore
+  -- we just mark then inactive for the next time darktable starts
+
+  -- deactivate it....
+
+  dt.preferences.write("script_manager", script, "bool", false)
+  dt.print_log("setting " .. scriptname .. " to not start")
+  dt.print(scriptname .. " will not be active when darktable is restarted")
+end
+
+local function create_enable_disable_button(btext, sname, req)
+  return dt.new_widget("button")
+  {
+    label = btext .. sname,
+    tooltip = get_script_doc(req),
+    clicked_callback = function (self)
+      -- split the label into action and target
+      local action, target = string.match(self.label, "(.+) (.+)")
+      -- load the script if it's not loaded
+      dt.print_log("Looking for target " .. target)
+      local scat = sm.category_selector.value
+      -- check, and fix, the filename for a - since it is a lua magic character in string match
+      dt.print_log("checking category " .. scat .. " for target " .. target)
+      local starget = du.join({scat, target}, "/")
+      dt.print_log("starget to activate is " .. starget)
+      dt.print_log("target to activate is " .. target)
+      if action == "Enable" then
+        local status = activate(starget, target)
+        if status then
+          self.label = "Disable " .. target
+        end
+      else
+        deactivate(starget, target)
+        self.label = "Enable " .. target
+      end
+    end
+  }
+end
+
+local function load_script_stack()
+  -- load the scripts
+  table.sort(sm.script_categories)
+  for _,cat in ipairs(sm.script_categories) do
+    local tmp = {}
+    table.sort(sm.script_names[cat])
+    if not sm.script_widgets[cat] then
+      for _,sname in ipairs(sm.script_names[cat]) do
+        local req = du.join({cat, sname}, "/")
+        local btext = "Enable "
+        if dt.preferences.read("script_manager", req, "bool") then
+          local status, err = prequire(sm.script_paths[req])
+          if status then 
+            btext = "Disable "
+          else
+            dt.print_error("Error loading " .. sname)
+            dt.print_error("Error message: " .. err)
+          end
+        else
+          dt.preferences.write("script_manager", req, "bool", false)
+        end
+        tmp[#tmp + 1] = create_enable_disable_button(btext, sname, req)
+      end
+
+      sm.script_widgets[cat] = dt.new_widget("box")
+      {
+        orientation = "vertical",
+        table.unpack(tmp),
+      }
+    elseif #sm.script_widgets[cat] ~= #sm.script_names[cat] then
+      for index,sname in ipairs(sm.script_names[cat]) do
+        local req = du.join({cat, sname}, "/")
+        dt.print_error("script is " .. sname .. " and index is " .. index)
+        if sm.script_widgets[cat][index] then
+          sm.script_widgets[cat][index] = nil
+        end
+        sm.script_widgets[cat][index] = create_enable_disable_button("Enable ", sname, req)
+      end
+    end
+  end
+  if not sm.script_stack then
+    sm.script_stack = dt.new_widget("stack"){}
+    for i,cat in ipairs(sm.script_categories) do
+      sm.script_stack[i] = sm.script_widgets[cat]
+    end
+    sm.script_stack.active = 1
+  end
+end
+
+local function update_stack_choices(combobox, choice_table)
+  sm.have_downloads = true
+  local items = #combobox
+  local choices = #choice_table
+  if #sm.script_widgets["downloads"] == 0 then
+    choices = choices - 1
+    sm.have_downloads = false
+  end
+  cnt = 1
+  for i, name in ipairs(choice_table) do 
+    if (name == "downloads" and sm.have_downloads) or name ~= "downloads" then
+      combobox[cnt] = name
+      cnt = cnt + 1
+    end
+  end
+  if choices < items then
+    for j = items, choices + 1, -1 do
+      combobox[j] = nil
+    end
+  end
+  combobox.value = 1
+end
+
+local function build_scripts_block()
+  -- build the whole script block
+  scan_scripts()
+
+    -- set up the stack for the choices
+  load_script_stack()
+
+  if not sm.category_selector then
+    -- set up the combobox for the categories
+
+    sm.category_selector = dt.new_widget("combobox"){
+      label = "Category",
+      tooltip = "Select the script category",
+      value = 1, "placeholder",
+      changed_callback = function(self)
+        local cnt = 1
+        for i,cat in ipairs(sm.script_categories) do
+          if cat == self.value then
+            sm.script_stack.active = i
+          end
+        end
+      end
+    }
+  end
+
+  update_stack_choices(sm.category_selector, sm.script_categories)
+
+  if not sm.scripts then
+    sm.scripts = dt.new_widget("box"){
+      orientation = "vertical",
+      dt.new_widget("label"){ label = "Scripts" },
+      sm.category_selector,
+      sm.script_stack,
+    }
+  end
+end
+
+local function insert_scripts_block()
+  table.insert(sm.main_menu_choices, "Enable/Disable Scripts")
+  update_combobox_choices(sm.main_menu, sm.main_menu_choices, 1)
+  sm.main_stack[#sm.main_stack + 1] = sm.scripts
+end
+
+local function use_lua_scripts_version()
+  if dt.configuration.running_os == "windows" then
+    -- copy tools\script_manger.lua to luarc
+    df.file_copy(LUA_DIR .. PS .. "tools\\script_manager.lua", dt.configuration.config_dir .. PS .. "luarc")
+  else
+    -- create a symbolic link from luarc to  tools/script_manager.lua
+    if df.check_if_file_exists(dt.configuration.config_dir .. "/luarc") then
+      os.remove(dt.configuration.config_dir .. "/luarc")
+    end
+    os.execute("ln -s " .. LUA_DIR .. "/tools/script_manager.lua " .. dt.configuration.config_dir .. "/luarc")
+  end
+end
+
+local function link_downloads_directory()
+  if not df.check_if_file_exists("$HOME/Downloads") then
+    os.execute("mkdir $HOME/Downloads")
+  end
+  if df.check_if_file_exists(LUA_DIR .. "/downloads") then
+    os.remove(LUA_DIR .. "/downloads")
+  end
+  os.execute("ln -s " .. "$HOME/Downloads " .. LUA_DIR .. "/downloads")
+end
+
+-- - - - - - - - - - - - - - - - - - - - - - - - 
+-- M A I N  P R O G R A M
+-- - - - - - - - - - - - - - - - - - - - - - - - 
+
+-- api check
+
+du.check_min_api_version("5.0.0")
+
+-- set up tables to contain all the widgets and choices
+
+sm.script_widgets = {}
+sm.script_categories = {}
+sm.script_names = {}
+sm.script_paths = {}
+sm.main_menu_choices = {}
+sm.main_stack_items = {}
+
+-- see if we've run this before
+
+sm.initialized = dt.preferences.read("script_manager", "initialized", "bool")
+
+if not sm.initialized then
+  -- write out preferences
+  dt.preferences.write("script_manager", "lua_repository", "string", LUA_SCRIPT_REPO)
+  dt.preferences.write("script_manager", "use_distro_version", "bool", false)
+  dt.preferences.write("script_manager", "link_downloads_directory", "bool", false)
+  dt.preferences.write("script_manager", "initialized", "bool", true)
+end
+
+sm.have_scripts = df.check_if_file_exists(LUA_DIR)
+
+sm.git_managed = df.check_if_file_exists(LUA_DIR .. PS .. ".git")
+
+if sm.have_scripts then
+  dt.print_log("found lua scripts directory")
+else
+  dt.print_log("lua scripts directory not found")
+end
+
+if sm.git_managed then
+  dt.print_log("scripts managed by git")
+else
+  dt.print_log("scripts not managed")
+end
+
+local git = df.check_if_bin_exists("git")
+
+if git then
+  dt.print_log("git found at " .. git)
+  sm.need_git = false
+else
+  dt.print_log("git not found")
+  sm.need_git = true
+end
+
+sm.repository = dt.new_widget("entry")
+{
+  text = dt.preferences.read("script_manager", "lua_repository", "string"),
+  editable = true,
+}
+
+sm.need_install = false
+
+local install_update_text = _("install")
+if sm.have_scripts and sm.git_managed then
+  install_update_text = _("update")
+else
+  sm.need_install = true
+end
+
+sm.install_update_button = dt.new_widget("button"){
+  label = install_update_text .. _(" scripts"),
+  clicked_callback = function(self)
+    sm.lua_repository = sm.repository.text
+    dt.preferences.write("script_manager", "lua_repository", "string", sm.repository.text)
+    if sm.need_install then
+      local result = install_scripts()
+      if result then
+        build_scripts_block()
+        insert_scripts_block()
+        sm.have_scripts = true
+        sm.git_managed = true
+        sm.need_install = false
+        self.label = _("update scripts")
+        dt.print(_("installed scripts from " .. sm.repository.text))
+      else
+        dt.print(_("Error installing scripts from " .. sm.repository.text))
+      end
+    else
+      local result = update_scripts()
+      if result then
+        --build_scripts_block()
+        dt.print(_("updated scripts from " .. sm.repository.text))
+      else
+        dt.print(_("Error updating scripts from " .. sm.repository.text))
+      end
+    end
+  end
+}
+
+if not sm.need_install then
+  sm.reinstall_button = dt.new_widget("button"){
+    label = "reinstall scripts",
+    clicked_callback = function(self)
+      sm.lua_repository = sm.repository.text
+      dt.preferences.write("script_manager", "lua_repository", "string", sm.repository.text)
+      local result = install_scripts()
+      if result then
+        build_scripts_block()
+        insert_scripts_block()
+        sm.have_scripts = true
+        sm.git_managed = true
+        sm.need_install = false
+        sm.install_update_button.label = "update scripts"
+        dt.print(_("reinstalled scripts from " .. sm.repository.text))
+        dt.print_log(_("scripts reinstalled"))
+      else
+        dt.print(_("ERROR: script reinstallation failed"))
+        dt.print_error(_("script reinstall failed"))
+      end
+    end
+  }
+
+  sm.install_update_widgets = {
+    sm.install_update_button,
+    sm.reinstall_button,
+  }
+else
+  sm.install_update_widgets = {
+    sm.install_update_button,
+  }
+end
+
+sm.install_update_box = dt.new_widget("box"){
+  orientation = "vertical",
+  dt.new_widget("label"){ label = "Install/Update scripts" },
+  table.unpack(sm.install_update_widgets),
+}
+
+table.insert(sm.main_menu_choices, "Install/Update Scripts")
+table.insert(sm.main_stack_items, sm.install_update_box)
+
+-- configuration items
+
+sm.repository_update = dt.new_widget("button"){
+  label = "update",
+  clicked_callback = function()
+    dt.preferences.write("script_manager", "lua_repository", "string", sm.repository.text)
+    sm.install_update_button.label = "install"
+    sm.install_update_button.sensitive = true
+    sm.reinstall_button.sensitive = false
+    sm.need_install = true
+  end
+}
+
+sm.repository_reset = dt.new_widget("button"){
+  label = "reset",
+  clicked_callback = function()
+    sm.repository.text = LUA_SCRIPT_REPO
+    dt.preferences.write("script_manager", "lua_repository", "string", LUA_SCRIPT_REPO)
+    sm.install_update_button.label = "install"
+    sm.install_update_button.sensitive = true
+    sm.reinstall_button.sensitive = false
+    sm.need_install = true
+  end
+}
+
+sm.update_reset = dt.new_widget("box"){
+  orientation = "horizontal",
+  sm.repository_update,
+  sm.repository_reset,
+}
+
+-- replace standalone version of script_manager with distributed version, i.e. tools/script_manager.lua
+  -- link on linux/MacOS, copy on windows
+  -- this option won't be active until the repository version of this script is accepted
+
+sm.use_lua_scripts_version = dt.new_widget("check_button"){
+  label = "Use lua scripts distributed version",
+  tooltip = "Use the standalone version (false) or the distributed version (true)",
+  value = dt.preferences.read("script_manager", "use_distro_version", "bool"),
+  clicked_callback = function(self)
+    if dt.preferences.read("script_manager", "use_distro_version", "bool") == self.value then
+      -- do nothing
+    else
+      sm.apply_configuration.sensitive = true
+    end
+  end
+}
+
+-- link downloads to $HOME/downloads on linux and MacOS
+
+sm.link_downloads_directory = dt.new_widget("check_button"){
+  label = "Link lua/downloads to $HOME/Downloads",
+  tooltip = "Linking the directories enables dropping a script in $HOME/downloads\nand having it recognized the next time darktable starts",
+  value = dt.preferences.read("script_manager", "link_downloads_directory", "bool"),
+  clicked_callback = function(self)
+    if dt.preferences.read("script_manager", "link_downloads_directory", "bool") == self.value then
+      -- do nothing
+    else
+      sm.apply_configuration.sensitive = true
+    end
+  end
+}
+
+sm.apply_configuration = dt.new_widget("button"){
+  label = "Apply",
+  sensitive = false,
+  clicked_callback = function(self)
+    dt.preferences.write("script_manager", "use_distro_version", "bool", sm.use_lua_scripts_version.value)
+    if sm.use_lua_scripts_version.value then
+      use_lua_scripts_version()
+    end
+    if dt.configuration.running_os ~= "windows" then
+      dt.preferences.write("script_manager", "link_downloads_directory", "bool", sm.link_downloads_directory.value)
+      if sm.link_downloads_directory.value then
+        link_downloads_directory()
+      end
+    end
+  end
+}
+
+-- get git location on windows
+
+sm.git_location = df.executable_path_widget({"git"})
+
+sm.configuration_widgets = {
+  sm.repository,
+  sm.update_reset,
+  dt.new_widget("separator"){},
+  dt.new_widget("separator"){},
+}
+
+if dt.configuration.running_os == "windows" or sm.need_git then
+  sm.configuration_widgets[#sm.configuration_widgets + 1] = sm.git_location
+end
+
+if dt.configuration.running_os ~= "windows" then
+  sm.configuration_widgets[#sm.configuration_widgets + 1] = sm.use_lua_scripts_version
+  sm.configuration_widgets[#sm.configuration_widgets + 1] = sm.link_downloads_directory
+end
+sm.configuration_widgets[#sm.configuration_widgets + 1] = sm.apply_configuration
+
+sm.config_box = dt.new_widget("box"){
+  orientation = "vertical",
+  dt.new_widget("label") { label = "Configuration" },
+  table.unpack(sm.configuration_widgets),
+}
+
+
+table.insert(sm.main_menu_choices, "Configure")
+table.insert(sm.main_stack_items, sm.config_box)
+
+-- set up the outside stack for config, install/update, and download
+
+  -- make a stack for the choices
+
+sm.main_stack = dt.new_widget("stack"){
+  table.unpack(sm.main_stack_items),
+}
+
+  -- make a combobox for the selector
+
+sm.main_menu = dt.new_widget("combobox"){
+  label = "Action",
+  tooltip = "Select the action you want to perform",
+  value = 1, "No actions available",
+  changed_callback = function(self)
+    for pos,str in ipairs(sm.main_menu_choices) do
+      if self.value == str then
+        sm.main_stack.active = pos
+        dt.preferences.write("script_manager", "sm_main_menu_value", "integer", pos)
+      end
+    end
+  end
+}
+
+if #sm.main_menu_choices > 0 then
+  update_combobox_choices(sm.main_menu, sm.main_menu_choices, 1)
+end
+
+sm.main_box = dt.new_widget("box"){
+  orientation = "vertical",
+  sm.main_menu,
+  sm.main_stack,
+}
+
+
+-- - - - - - - - - - - - - - - - - - - - - - - - 
+-- D A R K T A B L E  I N T E G R A T I O N 
+-- - - - - - - - - - - - - - - - - - - - - - - - 
+
+-- register the module
+dt.register_lib(
+  "script_manager",     -- Module name
+  "script manager",     -- Visible name
+  true,                -- expandable
+  false,               -- resetable
+  {[dt.gui.views.lighttable] = {"DT_UI_CONTAINER_PANEL_LEFT_BOTTOM", 100}},   -- containers
+  dt.new_widget("box") -- widget
+  {
+    orientation = "vertical",
+    sm.main_box,
+  },
+  nil,-- view_enter
+  nil -- view_leave
+)
+
+-- set up the scripts block if we have them otherwise we'll wait until we download them
+
+if sm.have_scripts then
+
+  -- scan for scripts and populate the categories
+  build_scripts_block()
+
+  -- add the widgets to the lib
+  insert_scripts_block()
+
+  sm.main_menu.selected = 3
+
+end
+
+collectgarbage("restart")