February 24, 2018

G’MIC 2.2 : New features and filters!

The IMAGE team of the GREYC laboratory (UMR CNRS 6072, Caen, France) is pleased to announce the release of a new 2.2 version of G’MIC, its open-source, generic, and extensible framework for image processing. As we already did in the past, we take this opportunity to look at the latest notable features added since the previous major release (2.0, last June).



Note 1: click on a picture to view a larger version.
Note 2: This is a translation of an original article, in French, published on Linuxfr.

1. Context and recent evolutions

G’MIC is a free and open-source software developed since August 2008 (distributed under the CeCILL license), by folks in the IMAGE team at the GREYC, a French public research laboratory located in Caen and supervised by three institutions: the CNRS, the University of Caen, and the ENSICAEN engineering school. This team is made up of researchers and lecturers specialized in the fields of algorithms and mathematics for image processing.
As one of the main developer of G’MIC, I wanted to sum up the work we’ve made on this software during these last months.

G'MIC logo
Fig. 1.1: The G’MIC project logo, and its cute little mascot “Gmicky” (designed by David Revoy).

G’MIC is multi-platform (GNU/Linux, MacOS, Windows …) and provides many ways of manipulating generic image data, i.e. still images or image sequences acquired as hyperspectral 2D or 3D floating-point arrays (including usual color images). More than 950 different image processing functions are already available in the G’MIC framework, this number being expandable through the use of the G’MIC scripting capabilities.

G'MIC plugin for GIMP
Fig.1.2: The G’MIC-Qt plugin for GIMP, currently the most popular G’MIC interface.

Since the last major version release there have been two important events in the project life:

1.1. Port of the G’MIC-Qt plugin to Krita

When we released version 2.0 of G’MIC a few months ago, we were happy to announce a complete rewrite (in Qt) of the plugin code for GIMP. An extra step has been taken, since this plugin has been extended to fit into the open-source digital painting software Krita.
This has been made possible thanks to the development work of Boudewijn Rempt (maintainer of Krita) and Sébastien Fourey (developer of the plugin). The G’MIC-Qt plugin is now available for Krita versions 3.3+ and, although it does not yet implement all the I/O functionality of its GIMP counterpart, the feedback we’ve had so far is rather positive.
This new port replaces the old G’MIC plugin for Krita which has not been maintained for some time. The good news for Krita users (and developers) is that they now have an up-to-date plugin whose code is common with the one running in GIMP and for which we will be able to ensure the maintenance and further developments.
Note this port required the writing of a source file host_krita.cpp (in C++) implementing the communication between the host software and the plugin, and it is reasonable to think that a similar effort would allow other programs to get their own version of the G’MIC plugin (and the 500 image filters that come with it!).

G'MIC for Krita
Fig. 1.3: Overview of the G’MIC-Qt plugin running on Krita.

1.2. CeCILL-C, a more permissive license

Another major event concerns the new license of use : The CeCILL-C license (that is in the spirit of the LGPL) is now available for some components of the G’MIC framework. This license is more permissive than the previously proposed CeCILL license (which is GPL-compatible) and is more suitable for the distribution of software libraries. This license extension (now double licensing) applies precisely to the core files of G’MIC, i.e. its C++ library libgmic. Thus, the integration of the libgmic features (therefore, all G’MIC image filters) is now allowed in software that are not themselves licensed under GPL/CeCILL (including closed source products).
The source code of the G’MIC-Qt plugin, meanwhile, remains distributed under the single CeCILL license (GPL-like).

2. Fruitful collaboration with David Revoy

If you’ve followed us for a while, you may have noticed that we very often refer to the work of illustrator David Revoy for his multiple contributions to G’MIC: mascot design, ideas of filters, articles or video tutorials, tests of all kinds, etc. More generally, David is a major contributor to the world of free digital art, as much with the comic Pepper & Carrot he produces (distributed under free license CC -BY), as with his suggestions and ongoing bug reports for the open-source software he uses.
Therefore, it seems quite natural to devote a special section to him in this article, summarizing the different ideas, contributions and experiments he has brought to G’MIC just recently. A big thank you, David for your availability, the sharing of your ideas, and for all your work in general!

2.1. Improving the lineart colorization filter

Let’s first mention the progress made on the Black & White / Colorize lineart (smart-coloring) filter that had appeared at the time of the 2.0 G’MIC release.
This filter is basically a lineart colorization assistant which was developed in collaboration with David. It tries to automatically generate a colorization layer for a given lineart, from the analysis of the contours and the geometry of that lineart. Following David‘s suggestions, we were able to add a new colorization mode, named “Autoclean“. The idea is to try to automatically “clean” a coloring layer (made roughly by the user) provided in addition to the lineart layer, using the same geometric analysis as for the previous colorization modes.
The use of this new mode is illustrated below, where a given lineart (left) has been colorized approximately by the user. From the two layers line art + color layer, our “Autoclean” algorithm generates an image (right), where the colors do not overflow the lineart contours (even for “virtual” contours that are not closed). The result is not always perfect, but nevertheless reduces the time spent in the tedious process of colorization.

Gmic_autoclean
Fig. 2.1: The new “Autoclean” mode of the lineart colorization filter can automatically “clean” a rough colorization layer.

Note that this filter is also equipped with a new hatch detection module, which makes it possible to avoid generating too many small areas when using the previously available random colorization mode, particularly when the input lineart contains a large number of hatches (see figure below).

Gmic_hatch_detect
Fig. 2.2: The new hatching detection module limits the number of small colored areas generated by the automatic random coloring mode.

2.2. Color equalizer in HSI, HSL and HSV spaces

More recently, David suggested the idea of a filter to separately vary the hue and saturation of colors having certain levels of luminosity. The underlying idea is to give the artist the ability to draw or paint digitally using only grayscale, then colorize his masterpiece afterwards by re-assigning specific colors to the different gray values of the image. The obtained result has of course a limited color range, but the overall color mood is already in place. The artist only has to retouch the colors locally rather than having to colorize the entire painting by hand.
The figure below illustrates the use of this new filter Colors/Equalize HSI/HSL/HSV available in the G’MIC plugin : each category of values can be finely adjusted, resulting in preliminary colorizations of black and white paintings.

Equalize HSI1
Equalize HSI2
Equalize HSI3
Fig. 2.3: Equalization in HSI/HSL/HSV colorspaces allows to easily set the global color mood for B&W paintings.

Note that the effect is equivalent to applying a color gradient to the different gray values of the image. This is something that could already be done quite easily in GIMP. But the main interest here is we can ensure that the pixel brightness remains unchanged during the color transformation, which is not an obvious property to preserve when using a gradient map.
What is nice about this filter is that it can apply to color photographs as well. You can change the hue and saturation of colors with a certain brightness, with an effect that can sometimes be surprising, like with the landscape photography shown below.

Equalize HSI4
Fig. 2.4: The filter “Equalize HSI/HSL/HSV” applied on a color photograph makes it possible to change the colorimetric environment, here in a rather extreme way.

2.3. Angular deformations

Another one of the David‘s ideas concerned the development of a random local deformation filter, having the ability to generate angular deformations. From an algorithmic point of view, it seemed relatively simple to achieve.
Note that once the implementation has been done (in concise style: 12 lines!) and pushed into the official filter updates, David just had to press the “Update Filters” button of his G’MIC-Qt plug-in, and the new effect Deformations/Crease was there immediately for testing. This is one of the practical side of developing new filters using the G’MIC script language!

G'MIC Crease
Fig. 2.5: New effect “Crease” for local angular deformations.

However, I must admit I didn’t really have an idea on what this could be useful for in practice. But the good thing about cooperating with David is that HE knows exactly what he’s going to do with it! For instance, to give a crispy look to the edges of his comics, or for improving the render of his alien death ray.

G'MIC Crease 2
G'MIC Crease 3
Fig. 2.6: Using the G’MIC “Crease” filter for two real cases of artistic creation.

3. Filters, filters, filters…

David Revoy is not the only user of G’MIC: we sometimes count up to 900 daily downloads from the main project website. So it happens, of course, that other enthusiastic users inspire us new effects, especially during those lovely discussions that take place on our forum, kindly made available by the PIXLS.US community.

3.1. Bring out the details without creating “halos”

Many photographers will tell you that it is not always easy to enhance the details in digital photographs without creating naughty artifacts that often have to be masked manually afterwards. Conventional contrast enhancement algorithms are most often based on increasing the local variance of pixel lightness, or on the equalization of their local histograms. Unfortunately, these operations are generally done by considering neighborhoods with a fixed size and geometry, where each pixel of a neighborhood is always considered with the same weight in the statistical calculations related to these algorithms.
It is simpler and faster, but from a qualitative point of view it is not an excellent idea: we often get “halos” around contours that were already very contrasted in the image. This classic phenomenon is illustrated below with the application of the Unsharp mask filter (the one present by default in GIMP) on a part of a landscape image. This generates an undesirable “halo” effect at the frontier between the mountain and the sky (this is particularly visible in full resolution images).

G'MIC details filters
Fig. 3.1: Unwanted “halo” effects often occur with conventional contrast enhancement filters.

The challenge of the detail enhancement algorithms is to be able to analyze the geometry of the local image structures in a more fine way, to take into account geometry-adaptive local weights for each pixel of a given neighborhood. To make it simple, we want to create anisotropic versions of the usual enhancement methods, orienting them by the edges detected in the images.
Following this logic, we have added two new G’MIC filters recently, namely Details/Magic details and Details/Equalize local histograms, which try to better take the geometric content of the image into account for local detail enhancement (e.g. using the bilateral filter).

G'MIC magic details
G'MIC equalize local histograms
G'MIC equalize local histograms
Fig. 3.2: The new G’MIC detail enhancement filters.

Thus, the application of the new G’MIC local histogram equalization on the landscape image shown before gives something slightly different : a more contrasted result both in geometric details and colors, and reduced halos.

G'MIC magic details
G'MIC magic details
Fig. 3.3: Differences of results between the standard GIMP Unsharp Mask filter and the local histogram equalization of G’MIC, for details enhancement.

3.2. Different types of image deformations

New filters to apply geometric deformations on images are added to G’MIC on a regular basis, and this new major version 2.2 offers therefore a bunch of new deformation filters.
So let’s start with Deformations/Spherize, a filter which allows to locally distort an image to give the impression that it is projected on a 3D sphere or ellipsoid. This is the perfect filter to turn your obnoxious office colleague into a Mr. Potato Head!

G'MIC spherize
G'MIC spherize
Fig .3.4: Two examples of 3D spherical deformations obtained with the G’MIC “Spherize” filter.

On the other hand, the filter Deformations/Square to circle implements the direct and inverse transformations from a square domain (or rectangle) to a disk (as mathematically described on this page), which makes it possible to generate this type of deformations.

G'MIC square to circle
Fig. 3.5: Direct and inverse transformations from a square domain to a disk.

The effect Degradations/Streak replaces an image area masked by the user (filled with a constant color) with one or more copies of a neighboring area. It works mainly as the GIMP clone tool but prevents the user to fill the entire mask manually.

G'MIC streak
Fig. 3.6: The “Streak” filter clones parts of the image into a user-defined color mask.

3.3. Artistic Abstractions

You might say that image deformations are nice, but sometimes you want to transform an image in a more radical way. Let’s introduce now the new effects that turn an image into a more abstract version (simplification and re-rendering). These filters have in common the analysis of the local image geometry, followed by a step of image synthesis.

For example, G’MIC filter Contours/Super-pixels locally gathers the image pixels with the same color to form a partitioned image, like a puzzle, with geometric shapes that stick to the contours. This partition is obtained using the SLIC method (Simple Linear Iterative Clustering), a classic image partitioning algorithm, which has the advantage of being relatively fast to compute.

G'MIC super pixels 1
G'MIC super pixels 2
Fig. 3.7: Decomposition of an image in super-pixels by the Simple Linear Iterative Clustering algorithm (SLIC).

The filter Artistic/Linify tries to redraw an input image by superimposing semi-transparent colored lines on an initially white canvas, as shown in the figure below. This effect is the re-implementation of the smart algorithm initially proposed on the site http://linify.me (initially implemented in JavaScript).

G'MIC linify 1
G'MIC linify 2
Fig. 3.8: The “Linify” effect tries to redraw an image by superimposing only semi-transparent colored lines on a white canvas.

The effect Artistic/Quadtree variations first decomposes an image as a quadtree, then re-synthesize it by drawing oriented and plain ellipses on a canvas, one ellipse for each quadtree leaf. This renders a rather interesting “painting” effect. It is likely that with more complex shapes, even more attractive renderings could be synthesized. Surely an idea to keep in mind for the next filters update 🙂

G'MIC quadtree 1
G'MIC quadtree 2
Fig. 3.9: Decomposing an image as a quadtree allows to re-synthesize it by superimposing only plain colored ellipses.

3.4. “Are there any more?”

And now that you have processed so many beautiful pictures, why not arrange them in the form of a superb photo montage? This is precisely the role of the filter Arrays & tiles/Drawn montage, which allows to create a juxtaposition of photographs very quickly, for any kind of shapes.
The idea is to provide the filter with a colored template in addition to the serie of photographs (Fig.3.10a), and then to associate each photograph with a different color of the template (Fig.3.10b). Next, the arrangement is done automatically by G’MIC, by resizing the images so that they appear best framed within the shapes defined in the given montage template (Fig.3.10c).
We made a video tutorial illustrating the use of this specific filter.

G'MIC drawn montage
Fig. 3.10a: Step 1: The user draws the desired organization of the montage with shapes of different colors.

G'MIC drawn montage
Fig. 3.10b: Step 2: G’MIC’s “Drawn Montage” filter allows you to associate a photograph for each template color.

G'MIC drawn montage
Fig. 3.10c: Step 3: The photo montage is then automatically synthetized by the filter.

But let’s go back to more essential questions: have you ever needed to draw gears? No?! It’s quite normal, that’s not something we do everyday! But just in case, the new G’MIC filter Rendering/Gear will be glad to help, with different settings to adjust gear size, colors and number of teeth. Perfectly useless, so totally indispensable!

G'MIC drawn montage
Fig. 3.11: The Gear filter, running at full speed.

Need a satin texture right now? No?! Too bad, the filter Patterns / Satin could have been of a great help!

G'MIC satin
Fig. 3.12: G’MIC’s satin filter will make your life more silky.

And finally, to end up with the series of these “effects that are useless until we need them”, note the apparition of the new filter Degradations/JPEG artifacts which simulates the appearance of JPEG compression artifacts due to the quantization of the DCT coefficients encoding 8×8 image blocks (yes, you will get almost the same result saving your image as a JPEG file with the desired quality).

Simulate JPEG Artifacts
Simulate JPEG Artifacts
Fig. 3.13: The “JPEG artifacts” filter simulates the image degradation due to 8×8 block DCT compression.

4. Other notable improvements

This review of these new available G’MIC filters should not overshadow the various improvements that have been made “under the hood” and that are equally important, even if they are less visible in practice for the user.

4.1. A better G’MIC-Qt plugin interface

A big effort of cleaning and restructuring the G’MIC-Qt plugin code has been realized, with a lot of little inconsistencies fixed in the GUI. Let’s also mention in bulk order some new interesting features that have appeared in the plugin:

  • The ability to set a timeout when trying to preview some computationnaly intensive filters.
  • A better management of the input-output parameters for each filter (with persistence, better menu location, and a reset button).
  • Maximizing the size of the preview area is now easier. Editing its zoom level manually is now possible, as well as chosing the language of the interface (regardless of the language used for the system), etc.

All these little things gathered together globally improves the user experience.

G'MIC Preferences
Fig. 4.1: Overview of the G’MIC-Qt plugin interface in its latest version 2.2.

4.2. Improvements in the G’MIC core

Even less visible, but just as important, many improvements have appeared in the G’MIC computational core and its associated G’MIC script language interpreter. You have to know that all of the available filters are actually written in the form of scripts in the G’MIC language, and each small improvement brought to the interpreter may have a beneficial consequence for all filters at once. Without going too much into the technical details of these internal improvements, we can highlight those points:

  • The notable improvement in the syntax of the language itself, which goes along with better performances for the analysis of the language syntax (therefore for the script executions), all this with a smaller memory footprint.
  • The G’MIC built-in mathematical expression evaluator is also experiencing various optimizations and new features, to consider even more possibilities for performing non-trivial operations at the pixel level.

  • A better support of raw video input/outputs (.yuv format) with support for4:2:2 and 4:4:4 formats, in addition to4:2:0 which was the only mode supported before.

  • Finally, two new animations have been added to the G’MIC demos menu (which is displayed e.g. when invoking gmic without arguments from the command-line):

    • First, a 3D starfield animation:

    Starfield demo
    Fig.4.2: New 3D starfield animation added to the G’MIC demo menu.

    Hanoi Demo
    Fig. 4.3: The playable 3D version of the “Tower of Hanoi”, available in G’MIC.

  • Finally, let us mention the introduction of the command tensors3d dedicated to the 3D representation of second order tensor fields. In practice, it does not only serve to make you want to eat Smarties®! It can be used for example to visualize certain regions of MRI volumes of diffusion tensors:

Tensors3d
Fig. 4.4: G’MIC rendering of a 3D tensor field, with command tensors3d.

4.3. New design for G’MIC Online

To finish this tour, let us also mention the complete redesign of G’MIC Online during the year 2017, done by Christophe Couronne and Véronique Robert from the development departement of the GREYC laboratory.
G’MIC Online is a web service allowing you to apply a subset of G’MIC filters on your images, directly inside a web browser. These web pages now have a responsive design, which makes them more enjoyable than before on mobile devices (smartphones and tablets). Shown below is a screenshot of this service running in Chrome/Android, on a 10” tablet.

G'MICol
Fig. 4.5: New responsive design of the G’MIC Online web service, running here on a 10″ tablet.

5. Conclusion and perspectives

The overview of this new version 2.2 of G’MIC is now over.
One possible conclusion could be: “There are plenty of perspectives!“.

G’MIC is a free project that can be considered as mature: the first lines of code were composed almost ten years ago, and today we have a good idea of the possibilities (and limits) of the beast. We hope to see more and more interest from FOSS users and developers, for example for integrating the G’MIC-Qt generic plugin in various software focused on image or video processing.

The possibility of using the G’MIC core under a more permissive CeCILL-C license can also be a source of interesting collaborations in the future (some companies have already approached us about this). While waiting for potential collaborations, we will do our best to continue developping G’MIC and feed it with new filters and effects, according to the suggestions of our enthusiastic users. A big thanks to them for their help and constant encouragement (the motivation to write code or articles, past 11pm, would not be the same without them!).

“Long live to open-source image processing and artistic creation!”

February 23, 2018

PEEC Planetarium Show: "The Analemma Dilemma"

[Analemma by Giuseppe Donatiello via Wikimedia Commons] Dave and I are giving a planetarium show at PEEC tonight on the analemma.

I've been interested in the analemma for years and have written about it before, here on the blog and in the SJAA Ephemeris. But there were a lot of things I still didn't understand as well as I liked. When we signed up three months ago to give this talk, I had plenty of lead time to do more investigating, uncovering lots of interesting details regarding the analemmas of other planets, the contributions of the two factors that go into the Equation of Time, why some analemmas are figure-8s while some aren't, and the supposed "moon analemmas" that have appeared on the Astronomy Picture of the Day. I added some new features to the analemma script I'd written years ago as well as corresponding with an expert who'd written some great Equation of Time code for all the planets. It's been fun.

I'll write about some of what I learned when I get a chance, but meanwhile, people in the Los Alamos area can hear all about it tonight, at our PEEC show: The Analemma Dilemma, 7 pm tonight, Friday Feb 23, at the Nature Center, admission $6/adult, $4/child.

February 21, 2018

G'MIC 2.2


G'MIC 2.2

New features and filters!

The IMAGE team of the GREYC laboratory (UMR CNRS 6072, Caen, France) is pleased to announce the release of a new 2.2 version of G’MIC, its open-source, generic, and extensible framework for image processing. As we already did in the past, we take this opportunity to look at the latest notable features added since the previous major release (2.0, last June).



Note 1: click on a picture to view a larger version. Note 2: This is a translation of an original article, in French, published on Linuxfr.

1. Context and recent evolutions

G’MIC is a free and open-source software developed since August 2008 (distributed under the CeCILL license), by folks in the IMAGE team at the GREYC, a French public research laboratory located in Caen and supervised by three institutions: the CNRS, the University of Caen, and the ENSICAEN engineering school. This team is made up of researchers and lecturers specialized in the fields of algorithms and mathematics for image processing. As one of the main developer of G’MIC, I wanted to sum up the work we’ve made on this software during these last months.

G'MIC logo Fig. 1.1: The G’MIC project logo, and its cute little mascot “Gmicky” (designed by David Revoy).

G’MIC is multi-platform (GNU/Linux, MacOS, Windows …) and provides many ways of manipulating generic image data, i.e. still images or image sequences acquired as hyperspectral 2D or 3D floating-point arrays (including usual color images). More than 950 different image processing functions are already available in the G’MIC framework, this number being expandable through the use of the G’MIC scripting capabilities.

G'MIC plugin for GIMP Fig.1.2: The G’MIC-Qt plugin for GIMP, currently the most popular G’MIC interface.

Since the last major version release there have been two important events in the project life:

1.1. Port of the G’MIC-Qt plugin to Krita

When we released version 2.0 of G’MIC a few months ago, we were happy to announce a complete rewrite (in Qt) of the plugin code for GIMP. An extra step has been taken, since this plugin has been extended to fit into the open-source digital painting software Krita. This has been made possible thanks to the development work of Boudewijn Rempt (maintainer of Krita) and Sébastien Fourey (developer of the plugin). The G’MIC-Qt plugin is now available for Krita versions 3.3+ and, although it does not yet implement all the I/O functionality of its GIMP counterpart, the feedback we’ve had so far is rather positive. This new port replaces the old G’MIC plugin for Krita which has not been maintained for some time. The good news for Krita users (and developers) is that they now have an up-to-date plugin whose code is common with the one running in GIMP and for which we will be able to ensure the maintenance and further developments. Note this port required the writing of a source file host_krita.cpp (in C++) implementing the communication between the host software and the plugin, and it is reasonable to think that a similar effort would allow other programs to get their own version of the G’MIC plugin (and the 500 image filters that come with it!).

G'MIC for Krita Fig. 1.3: Overview of the G’MIC-Qt plugin running on Krita.

1.2. CeCILL-C, a more permissive license

Another major event concerns the new license of use : The CeCILL-C license (that is in the spirit of the LGPL) is now available for some components of the G’MIC framework. This license is more permissive than the previously proposed CeCILL license (which is GPL-compatible) and is more suitable for the distribution of software libraries. This license extension (now double licensing) applies precisely to the core files of G’MIC, i.e. its C++ library libgmic. Thus, the integration of the libgmic features (therefore, all G’MIC image filters) is now allowed in software that are not themselves licensed under GPL/CeCILL (including closed source products). The source code of the G’MIC-Qt plugin, meanwhile, remains distributed under the single CeCILL license (GPL-like).

2. Fruitful collaboration with David Revoy

If you’ve followed us for a while, you may have noticed that we very often refer to the work of illustrator David Revoy for his multiple contributions to G’MIC: mascot design, ideas of filters, articles or video tutorials, tests of all kinds, etc. More generally, David is a major contributor to the world of free digital art, as much with the comic Pepper & Carrot he produces (distributed under free license CC -BY), as with his suggestions and ongoing bug reports for the open-source software he uses. Therefore, it seems quite natural to devote a special section to him in this article, summarizing the different ideas, contributions and experiments he has brought to G’MIC just recently. A big thank you, David for your availability, the sharing of your ideas, and for all your work in general!

2.1. Improving the lineart colorization filter

Let’s first mention the progress made on the Black & White / Colorize lineart (smart-coloring) filter that had appeared at the time of the 2.0 G’MIC release. This filter is basically a lineart colorization assistant which was developed in collaboration with David. It tries to automatically generate a colorization layer for a given lineart, from the analysis of the contours and the geometry of that lineart. Following David‘s suggestions, we were able to add a new colorization mode, named “Autoclean“. The idea is to try to automatically “clean” a coloring layer (made roughly by the user) provided in addition to the lineart layer, using the same geometric analysis as for the previous colorization modes. The use of this new mode is illustrated below, where a given lineart (left) has been colorized approximately by the user. From the two layers line art + color layer, our “Autoclean“ algorithm generates an image (right), where the colors do not overflow the lineart contours (even for “virtual” contours that are not closed). The result is not always perfect, but nevertheless reduces the time spent in the tedious process of colorization.

Gmic_autoclean Fig. 2.1: The new “Autoclean” mode of the lineart colorization filter can automatically “clean” a rough colorization layer.

Note that this filter is also equipped with a new hatch detection module, which makes it possible to avoid generating too many small areas when using the previously available random colorization mode, particularly when the input lineart contains a large number of hatches (see figure below).

Gmic_hatch_detect Fig. 2.2: The new hatching detection module limits the number of small colored areas generated by the automatic random coloring mode.

2.2. Color equalizer in HSI, HSL and HSV spaces

More recently, David suggested the idea of a filter to separately vary the hue and saturation of colors having certain levels of luminosity. The underlying idea is to give the artist the ability to draw or paint digitally using only grayscale, then colorize his masterpiece afterwards by re-assigning specific colors to the different gray values of the image. The obtained result has of course a limited color range, but the overall color mood is already in place. The artist only has to retouch the colors locally rather than having to colorize the entire painting by hand. The figure below illustrates the use of this new filter Colors/Equalize HSI/HSL/HSV available in the G’MIC plugin : each category of values can be finely adjusted, resulting in preliminary colorizations of black and white paintings.

Equalize HSI1 Equalize HSI2 Equalize HSI3 Fig. 2.3: Equalization in HSI/HSL/HSV colorspaces allows to easily set the global color mood for B&W paintings.

Note that the effect is equivalent to applying a color gradient to the different gray values of the image. This is something that could already be done quite easily in GIMP. But the main interest here is we can ensure that the pixel brightness remains unchanged during the color transformation, which is not an obvious property to preserve when using a gradient map. What is nice about this filter is that it can apply to color photographs as well. You can change the hue and saturation of colors with a certain brightness, with an effect that can sometimes be surprising, like with the landscape photography shown below.

Equalize HSI4 Fig. 2.4: The filter “Equalize HSI/HSL/HSV” applied on a color photograph makes it possible to change the colorimetric environment, here in a rather extreme way.

2.3. Angular deformations

Another one of the David‘s ideas concerned the development of a random local deformation filter, having the ability to generate angular deformations. From an algorithmic point of view, it seemed relatively simple to achieve. Note that once the implementation has been done (in concise style: 12 lines!) and pushed into the official filter updates, David just had to press the “Update Filters“ button of his G’MIC-Qt plug-in, and the new effect Deformations/Crease was there immediately for testing. This is one of the practical side of developing new filters using the G’MIC script language!

G'MIC Crease Fig. 2.5: New effect “Crease” for local angular deformations.

However, I must admit I didn’t really have an idea on what this could be useful for in practice. But the good thing about cooperating with David is that HE knows exactly what he’s going to do with it! For instance, to give a crispy look to the edges of his comics, or for improving the render of his alien death ray.

G'MIC Crease 2 G'MIC Crease 3 Fig. 2.6: Using the G’MIC “Crease” filter for two real cases of artistic creation.

3. Filters, filters, filters…

David Revoy is not the only user of G’MIC: we sometimes count up to 900 daily downloads from the main project website. So it happens, of course, that other enthusiastic users inspire us new effects, especially during those lovely discussions that take place on our forum, kindly made available by the PIXLS.US community.

3.1. Bring out the details without creating “halos”

Many photographers will tell you that it is not always easy to enhance the details in digital photographs without creating naughty artifacts that often have to be masked manually afterwards. Conventional contrast enhancement algorithms are most often based on increasing the local variance of pixel lightness, or on the equalization of their local histograms. Unfortunately, these operations are generally done by considering neighborhoods with a fixed size and geometry, where each pixel of a neighborhood is always considered with the same weight in the statistical calculations related to these algorithms. It is simpler and faster, but from a qualitative point of view it is not an excellent idea: we often get “halos” around contours that were already very contrasted in the image. This classic phenomenon is illustrated below with the application of the Unsharp mask filter (the one present by default in GIMP) on a part of a landscape image. This generates an undesirable “halo” effect at the frontier between the mountain and the sky (this is particularly visible in full resolution images).

G'MIC details filters Fig. 3.1: Unwanted “halo” effects often occur with conventional contrast enhancement filters.

The challenge of the detail enhancement algorithms is to be able to analyze the geometry of the local image structures in a more fine way, to take into account geometry-adaptive local weights for each pixel of a given neighborhood. To make it simple, we want to create anisotropic versions of the usual enhancement methods, orienting them by the edges detected in the images. Following this logic, we have added two new G’MIC filters recently, namely Details/Magic details and Details/Equalize local histograms, which try to better take the geometric content of the image into account for local detail enhancement (e.g. using the bilateral filter).

G'MIC magic details G'MIC equalize local histograms G'MIC equalize local histograms Fig. 3.2: The new G’MIC detail enhancement filters.

Thus, the application of the new G’MIC local histogram equalization on the landscape image shown before gives something slightly different : a more contrasted result both in geometric details and colors, and reduced halos.

G'MIC magic details G'MIC magic details Fig. 3.3: Differences of results between the standard GIMP Unsharp Mask filter and the local histogram equalization of G’MIC, for details enhancement.

3.2. Different types of image deformations

New filters to apply geometric deformations on images are added to G’MIC on a regular basis, and this new major version 2.2 offers therefore a bunch of new deformation filters. So let’s start with Deformations/Spherize, a filter which allows to locally distort an image to give the impression that it is projected on a 3D sphere or ellipsoid. This is the perfect filter to turn your obnoxious office colleague into a Mr. Potato Head!

G'MIC spherize G'MIC spherize Fig .3.4: Two examples of 3D spherical deformations obtained with the G’MIC “Spherize” filter.

On the other hand, the filter Deformations/Square to circle implements the direct and inverse transformations from a square domain (or rectangle) to a disk (as mathematically described on this page), which makes it possible to generate this type of deformations.

G'MIC square to circle Fig. 3.5: Direct and inverse transformations from a square domain to a disk.

The effect Degradations/Streak replaces an image area masked by the user (filled with a constant color) with one or more copies of a neighboring area. It works mainly as the GIMP clone tool but prevents the user to fill the entire mask manually.

G'MIC streak Fig. 3.6: The “Streak” filter clones parts of the image into a user-defined color mask.

3.3. Artistic Abstractions

You might say that image deformations are nice, but sometimes you want to transform an image in a more radical way. Let’s introduce now the new effects that turn an image into a more abstract version (simplification and re-rendering). These filters have in common the analysis of the local image geometry, followed by a step of image synthesis.

For example, G’MIC filter Contours/Super-pixels locally gathers the image pixels with the same color to form a partitioned image, like a puzzle, with geometric shapes that stick to the contours. This partition is obtained using the SLIC method (Simple Linear Iterative Clustering), a classic image partitioning algorithm, which has the advantage of being relatively fast to compute.

G'MIC super pixels 1 G'MIC super pixels 2 Fig. 3.7: Decomposition of an image in super-pixels by the Simple Linear Iterative Clustering algorithm (SLIC).

The filter Artistic/Linify tries to redraw an input image by superimposing semi-transparent colored lines on an initially white canvas, as shown in the figure below. This effect is the re-implementation of the smart algorithm initially proposed on the site http://linify.me (initially implemented in JavaScript).

G'MIC linify 1 G'MIC linify 2 Fig. 3.8: The “Linify” effect tries to redraw an image by superimposing only semi-transparent colored lines on a white canvas.

The effect Artistic/Quadtree variations first decomposes an image as a quadtree, then re-synthesize it by drawing oriented and plain ellipses on a canvas, one ellipse for each quadtree leaf. This renders a rather interesting “painting” effect. It is likely that with more complex shapes, even more attractive renderings could be synthesized. Surely an idea to keep in mind for the next filters update :)

G'MIC quadtree 1 G'MIC quadtree 2 Fig. 3.9: Decomposing an image as a quadtree allows to re-synthesize it by superimposing only plain colored ellipses.

3.4. “Are there any more?”

And now that you have processed so many beautiful pictures, why not arrange them in the form of a superb photo montage? This is precisely the role of the filter Arrays & tiles/Drawn montage, which allows to create a juxtaposition of photographs very quickly, for any kind of shapes. The idea is to provide the filter with a colored template in addition to the serie of photographs (Fig.3.10a), and then to associate each photograph with a different color of the template (Fig.3.10b). Next, the arrangement is done automatically by G’MIC, by resizing the images so that they appear best framed within the shapes defined in the given montage template (Fig.3.10c). We made a video tutorial illustrating the use of this specific filter.

G'MIC drawn montage Fig. 3.10a: Step 1: The user draws the desired organization of the montage with shapes of different colors.
G'MIC drawn montage Fig. 3.10b: Step 2: G’MIC’s “Drawn Montage” filter allows you to associate a photograph for each template color.
G'MIC drawn montage Fig. 3.10c: Step 3: The photo montage is then automatically synthetized by the filter.

But let’s go back to more essential questions: have you ever needed to draw gears? No?! It’s quite normal, that’s not something we do everyday! But just in case, the new G’MIC filter Rendering/Gear will be glad to help, with different settings to adjust gear size, colors and number of teeth. Perfectly useless, so totally indispensable!

G'MIC drawn montage Fig. 3.11: The Gear filter, running at full speed.

Need a satin texture right now? No?! Too bad, the filter Patterns / Satin could have been of a great help!

G'MIC satin Fig. 3.12: G’MIC’s satin filter will make your life more silky.

And finally, to end up with the series of these “effects that are useless until we need them”, note the apparition of the new filter Degradations/JPEG artifacts which simulates the appearance of JPEG compression artifacts due to the quantization of the DCT coefficients encoding 8×8 image blocks (yes, you will get almost the same result saving your image as a JPEG file with the desired quality).

Simulate JPEG Artifacts Simulate JPEG Artifacts Fig. 3.13: The “JPEG artifacts” filter simulates the image degradation due to 8×8 block DCT compression.

4. Other notable improvements

This review of these new available G’MIC filters should not overshadow the various improvements that have been made “under the hood” and that are equally important, even if they are less visible in practice for the user.

4.1. A better G’MIC-Qt plugin interface

A big effort of cleaning and restructuring the G’MIC-Qt plugin code has been realized, with a lot of little inconsistencies fixed in the GUI. Let’s also mention in bulk order some new interesting features that have appeared in the plugin:

  • The ability to set a timeout) when trying to preview some computationnaly intensive filters.
  • A better management of the input-output parameters for each filter (with persistence, better menu location, and a reset button).
  • Maximizing the size of the preview area is now easier. Editing its zoom level manually is now possible, as well as chosing the language of the interface (regardless of the language used for the system), etc.

All these little things gathered together globally improves the user experience.

G'MIC Preferences Fig. 4.1: Overview of the G’MIC-Qt plugin interface in its latest version 2.2.

4.2. Improvements in the G’MIC core

Even less visible, but just as important, many improvements have appeared in the G’MIC computational core and its associated G’MIC script language interpreter. You have to know that all of the available filters are actually written in the form of scripts in the G’MIC language, and each small improvement brought to the interpreter may have a beneficial consequence for all filters at once. Without going too much into the technical details of these internal improvements, we can highlight those points:

  • The notable improvement in the syntax of the language itself, which goes along with better performances for the analysis of the language syntax (therefore for the script executions), all this with a smaller memory footprint.
  • The G’MIC built-in mathematical expression evaluator is also experiencing various optimizations and new features, to consider even more possibilities for performing non-trivial operations at the pixel level.

  • A better support of raw video input/outputs (.yuv format) with support for4:2:2 and 4:4:4 formats, in addition to4:2:0 which was the only mode supported before.

  • Finally, two new animations have been added to the G’MIC demos menu (which is displayed e.g. when invoking gmic without arguments from the command-line):

    • First, a 3D starfield animation:
    Starfield demo Fig.4.2: New 3D starfield animation added to the G’MIC demo menu.
    Hanoi Demo Fig. 4.3: The playable 3D version of the “Tower of Hanoi”, available in G’MIC.
  • Finally, let us mention the introduction of the command tensors3d dedicated to the 3D representation of second order tensor fields. In practice, it does not only serve to make you want to eat Smarties®! It can be used for example to visualize certain regions of MRI volumes of diffusion tensors:

    Tensors3d Fig. 4.4: G’MIC rendering of a 3D tensor field, with command tensors3d.

4.3. New design for G’MIC Online

To finish this tour, let us also mention the complete redesign of G’MIC Online during the year 2017, done by Christophe Couronne and Véronique Robert from the development departement of the GREYC laboratory. G’MIC Online is a web service allowing you to apply a subset of G’MIC filters on your images, directly inside a web browser. These web pages now have a responsive design, which makes them more enjoyable than before on mobile devices (smartphones and tablets). Shown below is a screenshot of this service running in Chrome/Android, on a 10’’ tablet.

G'MICol Fig. 4.5: New responsive design of the G’MIC Online web service, running here on a 10” tablet.

5. Conclusion and perspectives

The overview of this new version 2.2 of G’MIC is now over. One possible conclusion could be: “There are plenty of perspectives!“.

G’MIC is a free project that can be considered as mature: the first lines of code were composed almost ten years ago, and today we have a good idea of the possibilities (and limits) of the beast. We hope to see more and more interest from FOSS users and developers, for example for integrating the G’MIC-Qt generic plugin in various software focused on image or video processing.

The possibility of using the G’MIC core under a more permissive CeCILL-C license can also be a source of interesting collaborations in the future (some companies have already approached us about this). While waiting for potential collaborations, we will do our best to continue developping G’MIC and feed it with new filters and effects, according to the suggestions of our enthusiastic users. A big thanks to them for their help and constant encouragement (the motivation to write code or articles, past 11pm, would not be the same without them!).

“Long live open-source image processing and artistic creation!”

February 20, 2018

CSS Grid

This would totally have been a tweet or a facebook post, but I’ve decided to invest a little more energy and post these on my blog, accessible to everybody. Getting old, I guess. We’re all mortal and the web isn’t open by its own.

In the past few days I’ve been learning about CSS grid while redesigning Flatpak and Flathub sites (still coming). And with the knowledge of really grokking only a fraction of it, I’m in love. So far I really dig:

  • Graceful fallback
  • Layout fully controlled by the theme
  • Controlled whitespace (meaning the layout won’t fall apart when you add or remove some whitespace)
  • Reasonable code legibility
  • Responsive layouts even without media queries

Whitespace on the Web

The fact that things are sized and defined very differently and getting grips with implicit sizing will take some time, but it seems to have all the answers to the problems I ran into so far. Do note that I never got super fluent in the flexbox, either.

I love the few video bites that Jen Simmons publishes periodically. The only downside to all this is seeing the mess with the legacy grid systems I have on the numerous websites like this one.

February 19, 2018

Interview with Christine Garner

Could you tell us something about yourself?

I’m 35 years old from Shropshire in Britain. I like comfy socks and tea.

I did Archaeology in University and I love history, mythology, folklore and nature. I’ve always been drawing from an early age. I graduated in 2003 with an archaeology degree. I taught myself digital art and web coding skills for fun and practical reasons. I used to do self-employed web design and admin type jobs, but in 2013 I became disillusioned with my life and had depression. I took a Foundation art course in 2013 deciding to pursue my artistic passions instead.

Since 2014 I’ve been practising art like crazy and building up a portfolio of illustration work.

Do you paint professionally, as a hobby artist, or both?

Both. I use digital painting to make studies for my illustration work and as a way to experiment. I would love to do paid freelance illustration work should the right opportunity come along. I am happy making my own projects as well, it helps me learn and get better.

What genre(s) do you work in?

I’ve found I like to draw and paint themes from nature and animals. I read about myths and folklore. I’ve been working on personal projects about mermaids and river goddesses. It’s still in the early stages. I’m developing my style to incorporate more fantasy subjects in the future and mix in my pattern work.

Whose work inspires you most — who are your role models as an artist?

I look at archaeology, art history and graphic design for inspiration. I don’t have a favourite artist at the moment. There are lots of styles I’m inspired by it’s difficult to choose one. I’ve been looking at folk art more and trying to loosen up with my painting.

How and when did you get to try digital painting for the first time?

I tried digital painting for the first time in 2004 when I wanted to texture my own skins for the Maxis Sims game I used to play. I learnt web coding as well so i could share my efforts with people. My sister had a copy of Photoshop 6 which I used, and I had a tiny non-sensitive drawing tablet my mum had given me. My first job as a data entry clerk in a bank (yawn) enabled me to save up and get my first proper Wacom tablet around 2005. I wished I could do something with my Archaeology instead but there were no paid local jobs. From looking on the Internet I saw people made amazing works of art with digital software. I got obsessed with concept art and fantasy art and tried to get better. I used to haunt the CGSociety and GFXArtist and I bought ImagineFX magazine to try and learn all I could on my own. There were not the resources back then that there are now, but it was fun and a good distraction from the rest of my life.

What makes you choose digital over traditional painting?

Digital painting is a versatile medium. When I was starting out it saved me money on buying expensive art materials, and I didn’t have any room for doing traditional painting in my small bedroom. I bought a student version of Corel Painter to start with, which was one of my only options back in 2005. This enabled me to try lots of different types of art styles without the mess. Now I have my own house, I have more room and I’ve collected more art materials I tend to mix using traditional and digital mediums depending on the effect I want. If I want to have the texture of coloured pencils for example, I’ve found it is easier to do it with coloured pencils first. Over the years I’ve tried a lot of different digital painting programs and learnt how to do vector art as well.

How did you find out about Krita?

I found out about Krita from surfing on the Internet around 2011. I’d been using MyPaint and heard about Krita from my geeky research. I’m a bit of a geek when it comes to digital art so I like trying different softwares.

What was your first impression?

The first time I tried it I was still a heavy Corel Painter user, but I bookmarked it for future reference seeing it had potential. I tried it again around 2013 and fell in love with it over using Corel Painter. The interface was great, it was much nicer and easier to use than Corel Painter, and it had lovely practical brushes out of the box.

What do you love about Krita?

I love the layer system and the way the brushes work. They are as sensitive as any in more expensive programs, and are as customisable. The mirror tool is fun and I like the seamless pattern feature. I’ve been so busy I haven’t gotten around to trying animation yet, which is a cool feature.

What do you think needs improvement in Krita? Is there anything that really annoys you?

I can’t think of anything which annoys me about Krita, unless it’s a fault of my own hardware setup. Sometimes I’ll have to minimise and maximise Krita to get pressure sensitivity back after using different programs while painting. It’s probably because I have an Ugee 2150 and I use Windows 10 which keeps doing annoying creator updates. Because I use other software as well (Photoshop CS3, Affinity Designer, Affinity Photo) any weaknesses in Krita can be overcome using those and vice versa.

What sets Krita apart from the other tools that you use?

I like the default brushes which come with Krita. In all the other programs I’ve used I have had to customise and make new brushes or buy brushes to get what I want. There is also a strong community feel about Krita and growing resources. I wrote a long article about why Krita is great for beginners to digital painting on Medium.

If you had to pick one favourite of all your work done in Krita so far, what would it be, and why?

The gold fish studies, because they were fun to do and turned out pretty well. I want to do more work in the same style.

What techniques and brushes did you use in it?

I used the default brushes. Most of them are by David Revoy. The digital sketch brush is one of my favourites, and the alchemy brush is useful for blocking in areas. First I blocked in the silhouette of the goldfish with an opaque round brush. Then I tool advantage of layer clipping masks to paint within the shape of the goldfish blocking in areas of colour with alchemy brush and the magnetic lasso tool. I also used gradient fills in the early stages to get nice base colour blending for painting the details into. I used different layer styles for shadows (multiply) and highlights (screen / overlay). Start with blocking in, big brushes and go smaller for the details.

Where can people see more of your work?

I have a website at https://thimblefolio.com.

I’m also on the following websites:

Behance: https://www.behance.net/chrisgarnerart
Dribbble: https://dribbble.com/chrisgarnerart
Instagram: https://www.instagram.com/thimblefoliopress/
Tumblr: https://thimblefolio.tumblr.com/
YouTube: https://www.youtube.com/user/cribbyGart/featured
Medium: https://medium.com/@thimblefolio
Society 6: https://society6.com/thimblefoliopress
Curioos: https://www.curioos.com/thimblefolio
Spoonflower: https://www.spoonflower.com/profiles/thimblefolio

Anything else you’d like to share?

Thank you very much to all the people who work on developing and improving Krita, and to everyone who creates tutorials and resources. Its great digital art is accessible for as many people as possible, learning digital painting is a positive and fun thing to do.

February 17, 2018

Multiplexing Input or Output on a Raspberry Pi Part 2: Port Expanders

In the previous article I talked about Multiplexing input/output using shift registers for a music keyboard project. I ended up with three CD4021 8-bit shift registers cascaded. It worked; but I found that I was spending all my time in the delays between polling each bit serially. I wanted a way to read those bits faster. So I ordered some I/O expander chips.

[Keyboard wired to Raspberry Pi with two MCP23017 port expanders] I/O expander, or port expander, chips take a lot of the hassle out of multiplexing. Instead of writing code to read bits serially, you can use I2C. Some chips also have built-in pullup resistors, so you don't need all those extra wires for pullups or pulldowns. There are lots of options, but two common chips are the MCP23017, which controls 16 lines, and the MCP23008 and PCF8574p, which each handle 8. I'll only discuss the MCP23017 here, because if eight is good, surely sixteen is better! But the MCP23008 is basically the same thing with fewer I/O lines.

A good tutorial to get you started is How To Use A MCP23017 I2C Port Expander With The Raspberry Pi - 2013 Part 1 along with part 2, Python and part 3, reading input.

I'm not going to try to repeat what's in those tutorials, just fill in some gaps I found. For instance, I didn't find I needed sudo for all those I2C commands in Part 1 since my user is already in the i2c group.

Using Python smbus

Part 2 of that tutorial uses Python smbus, but it doesn't really explain all the magic numbers it uses, so it wasn't obvious how to generalize it when I added a second expander chip. It uses this code:

DEVICE = 0x20 # Device address (A0-A2)
IODIRA = 0x00 # Pin direction register
OLATA  = 0x14 # Register for outputs
GPIOA  = 0x12 # Register for inputs

# Set all GPA pins as outputs by setting
# all bits of IODIRA register to 0
bus.write_byte_data(DEVICE,IODIRA,0x00)

# Set output all 7 output bits to 0
bus.write_byte_data(DEVICE,OLATA,0)

DEVICE is the address on the I2C bus, the one you see with i2cdetect -y 1 (20, initially).

IODIRA is the direction: when you call

bus.write_byte_data(DEVICE, IODIRA, 0x00)
you're saying that all eight bits in GPA should be used for output. Zero specifies output, one input: so if you said
bus.write_byte_data(DEVICE, IODIRA, 0x1F)
you'd be specifying that you want to use the lowest five bits for output and the upper three for input.

OLATA = 0x14 is the command to use when writing data:

bus.write_byte_data(DEVICE, OLATA, MyData)
means write data to the eight GPA pins. But what if you want to write to the eight GPB pins instead? Then you'd use
OLATB  = 0x15
bus.write_byte_data(DEVICE, OLATB, MyData)

Likewise, if you want to read input from some of the GPB bits, use

GPIOB  = 0x13
val = bus.read_byte_data(DEVICE, GPIOB)

The MCP23017 even has internal pullup resistors you can enable:

GPPUA  = 0x0c    # Pullup resistor on GPA
GPPUB  = 0x0d    # Pullup resistor on GPB
bus.write_byte_data(DEVICE, GPPUB, inmaskB)

Here's a full example: MCP23017.py on GitHub.

Using WiringPi

You can also talk to an MCP23017 using the WiringPi library. In that case, you don't set all the bits at once, but instead treat each bit as though it were a separate pin. That's easier to think about conceptually -- you don't have to worry about bit shifting and masking, just use pins one at a time -- but it might be slower if the library is doing a separate read each time you ask for an input bit. It's probably not the right approach to use if you're trying to check a whole keyboard's state at once.

Start by picking a base address for the pin number -- 65 is the lowest you can pick -- and initializing:

pin_base = 65
i2c_addr = 0x20

wiringpi.wiringPiSetup()
wiringpi.mcp23017Setup(pin_base, i2c_addr)

Then you can set input or output mode for each pin:

wiringpi.pinMode(pin_base, wiringpi.OUTPUT)
wiringpi.pinMode(input_pin, wiringpi.INPUT)
and then write to or read from each pin:
wiringpi.digitalWrite(pin_no, 1)
val = wiringpi.digitalRead(pin_no)

WiringPi also gives you access to the MCP23017's internal pullup resistors:

wiringpi.pullUpDnControl(input_pin, 2)

Here's an example in Python: MCP23017-wiringpi.py on GitHub, and one in C: MCP23017-wiringpi.c on GitHub.

Using multiple MCP23017s

But how do you cascade several MCP23017 chips?

Well, you don't actually cascade them. Since they're I2C devices, you wire them so they each have different addresses on the I2C bus, then query them individually. Happily, that's easier than keeping track of how many bits you've looped through ona shift register.

Pins 15, 16 and 17 on the chip are the address lines, labeled A0, A1 and A2. If you ground all three you get the base address of 0x20. With all three connected to VCC, it will use 0x27 (binary 111 added to the base address). So you can send commands to your first device at 0x20, then to your second one at 0x21 and so on. If you're using WiringPi, you can call mcp23017Setup(pin_base2, i2c_addr2) for your second chip.

I had trouble getting the addresses to work initially, and it turned out the problem wasn't in my understanding of the address line wiring, but that one of my cheap Chinese breadboard had a bad power and ground bus in one quadrant. That's a good lesson for the future: when things don't work as expected, don't assume the breadboard is above suspicion.

Using two MCP23017 chips with their built-in pullup resistors simplified the wiring for my music keyboard enormously, and it made the code cleaner too. Here's the modified code: keyboard.py on GitHub.

What about the speed? It is indeed quite a bit faster than the shift register code. But it's still too laggy to use as a real music keyboard. So I'll still need to do more profiling, and maybe find a faster way of generating notes, if I want to play music on this toy.

Look, new presets! Another Krita 4 development build!

We’ve been focusing like crazy on the Krita 4 release. We managed to close some 150 bugs in the past month, and Krita 4 is getting stable enough for many people to use day in, day out. There’s still more to be done, of course! So we’ll continue fixing issues and applying polish for at least another four weeks.

One of the things we’re doing as well is redesigning the set of default brush presets and brush tips that come with Krita. Brush tips are the little images one can paint with, and brush presets are the brushes you can select in the brush palette or brush popup. The combination of a tip, some settings and a smart bit of coding!

Our old set was fine, but it was based on David Revoy‘s earliest Krita brush bundles, and for Krita 4 we are revamping the entire set. We’ve added many new options to the brushes since then! So, many artists are working together to create a good-looking, useful and interesting brushes for Krita 4.

It’s still work in progress! But we want your feedback. So… Download the new Krita 4 development builds, and start doodling, sketching, painting, rendering… And then tell us what you think:

Do the brush preset survey!

Apart from the new brushes, and the bug fixes, there’s also news for Linux users: we updated our AppImage build scripts, and the new appimage includes Python scripting and the Touch UI docker. Note: by default all scripts are disabled, so you need to go to Settings/Configure Krita/Python scripts and enable the scripts you want to use.

Help with the release!

We’re having our hands full with tasks like coding, bug fixing, manual updating… More than full! One task that looks like it’s going to slip is creating a cool what-new-in-4 release video. So: if you’re good at creating videos and want to help the team, please join us on #krita on irc.freenode.net and help us out!

Windows

Note for Windows users: if you encounter crashes, please follow these instructions to use the debug symbols so we can figure out where Krita crashes. On Windows, gmic-qt is included.

Linux

On Linux, you need to download the gmic-qt appimage separately. You can configure the location of gmic-qt in Krita’s settings. Krita requires the “XDG_DATA_DIRS” environment variable to be set. Most distributions do this, but if yours doesn’t, set “XDG_DATA_DIRS” to “/usr/local/share/:/usr/share/” as a workaround. Next build Krita will do this by itself.

(If, for some reason, Firefox thinks it needs to load this as text: to download, right-click on the link.)

OSX/macOS

The minimum version of OSX/macOS is 10.11

Note: the python, gmic-qt and pdf plugins are not available on OSX.

md5sums

For all downloads:

Support Krita

Krita is a free and open source project. Please consider supporting the project with donations or by buying training videos or the artbook! With your support, we can keep the core team working on Krita full-time.

February 16, 2018

LVFS will block old versions of fwupd for some firmware

The ability to restrict firmware to specific versions of fwupd and the existing firmware version was added to fwupd in version 0.8.0. This functionality was added so that you could prevent the firmware being deployed if the upgrade was going to fail, either because:

  • The old version of fwupd did not support the new hardware quirks
  • If the upgraded-from firmware had broken upgrade functionality

The former is solved by updating fwupd, the latter is solved by following the vendor procedure to manually flash the hardware, e.g. using a DediProg to flash the EEPROM directly. Requiring a specific fwupd version is used by the Logitech Unifying receiver update for example, and requiring a previous minimum firmware version is used by one (soon to be two…) laptop OEMs at the moment.

Although fwupd 0.8.0 was released over a year ago it seems people are still downloading firmware with older fwupd versions. 98% of the downloads from the LVFS are initiated from gnome-software, and 2% of people using the fwupdmgr command line or downloading the .cab file from the LVFS using a browser manually.

At the moment, fwupd is being updated in Ubuntu xenial to 0.8.3 but it is still stuck at the long obsolete 0.7.4 in Debian stable. Fedora, or course, is 100% up to date with 1.0.5 in F27 and 0.9.6 in F26 and F25. Even RHEL 7.4 has 0.8.2 and RHEL 7.5 will be 1.0.1.

Detecting the fwupd version also gets slightly more complicated, as the user agent only gives us the ‘client version’ rather than the ‘fwupd version’ in most software. This means we have to use the minimum fwupd version required by the client when choosing if it is safe to provide the file. GNOME Software version 3.26.0 was the first version to depend on fwupd ≥ 0.8.0 and so anything newer than that would be safe. This gives a slight problem, as Ubuntu will be shipping an old gnome-software 3.20.x and a new-enough fwupd 0.8.x and so will be blacklisted for any firmware that requires a specific fwupd version. Which includes the Logitech security update…

The user agent we get from gnome-software is gnome-software/3.20.1 and so we can’t do anything very clever. I’m obviously erring on not bricking a tiny amount of laptop hardware rather than making a lot of Logitech hardware secure on Ubuntu 16.04, given the next LTS 18.04 is out on April 26th anyway. This means people might start getting a detected fwupd version too old message on the console if they try updating using 16.04.

A workaround for xenial users might be if someone at Canonical could include this patch that changes the user agent in gnome-software package to be gnome-software/3.20.1 fwupd/0.8.3 and I can add a workaround in the LVFS download code to parse that. Comments welcome.

February 13, 2018

Multiplexing Input or Output on a Raspberry Pi Part 1: Shift Registers

I was scouting for parts at a thrift shop and spotted a little 23-key music keyboard. It looked like a fun Raspberry Pi project.

I was hoping it would turn out to use some common protocol like I2C, but when I dissected it, it turned out there was a ribbon cable with 32 wires coming from the keyboard. So each key is a separate pushbutton.

[23-key keyboard wired to a Raspberry Pi] A Raspberry Pi doesn't have that many GPIO pins, and neither does an Arduino Uno. An Arduino Mega does, but buying a Mega to go between the Pi and the keyboard kind of misses the point of scavenging a $3 keyboard; I might as well just buy an I2C or MIDI keyboard. So I needed some sort of I/O multiplexer that would let me read 31 keys using a lot fewer pins.

There are a bunch of different approaches to multiplexing. A lot of keyboards use a matrix approach, but that makes more sense when you're wiring up all the buttons from scratch, not starting with a pre-wired keyboard like this. The two approaches I'll discuss here are shift registers and multiplexer chips.

If you just want to get the job done in the most efficient way, you definitely want a multiplexer (port expander) chip, which I'll cover in Part 2. But for now, let's look at the old-school way: shift registers.

PISO Shift Registers

There are lots of types of shift registers, but for reading lots of inputs, you need a PISO shift register: "Parallel In, Serial Out." That means you can tell the chip to read some number -- typically 8 -- of inputs in parallel, then switch into serial mode and read all the bits one at a time.

Some PISO shift registers can cascade: you can connect a second shift register to the first one and read twice as many bits. For 23 keys I needed three 8-bit shift registers.

Two popular cascading PISO shift registers are the CD4021 and the SN74LS165. They work similarly but they're not exactly the same.

The basic principle with both the CD4021 and the SN74LS165: connect power and ground, and wire up all your inputs to the eight data pins. You'll need pullup or pulldown resistors on each input line, just like you normally would for a pushbutton; I recommend picking up a few high-value (like 1-10k) resistor arrays: you can get these in SIP (single inline package) or DIP (dual-) form factors that plug easily into a breadboard. Resistor arrays can be either independent two pins for each resistor in the array) or bussed (one pin in the chip is a common pin, which you wire to ground for a pulldown or V+ for a pullup; each of the rest of the pins is a resistor). I find bussed networks particularly handy because they can reduce the number of wires you need to run, and with a job where you're multiplexing lots of lines, you'll find that getting the wiring straight is a big part of the job. (See the photo above to see what a snarl this was even with resistor networks.)

For the CD4021, connect three more pins: clock and data pins (labeled CLK and either Q7 or Q8 on the chip's pinout, pins 10 and 3), plus a "latch" pin (labeled M, pin 9). For the SN74LS165, you need one more pin: you need clock and data (labeled CP and Q7, pins 2 and 9), latch (labeled PL, pin 1), and clock enable (labeled CE, pin 15).

At least for the CD4021, some people recommend a 0.1 uF bypass capacitor across the power/ground connections of each CD4021.

If you need to cascade several chips with the CD4021, wire DS (pin 11) from the first chip to Q7 (pin 3), then wire both chips clock lines together and both chips' data lines together. The SN74LS165 is the same: DS (pin 10) to Q8 (pin 9) and tie the clock and data lines together.

Once wired up, you toggle the latch to read the parallel data, then toggle it again and use the clock pin to read the series of bits. You can see the specific details in my Python scripts: CD4021.py on GitHub and SN74LS165.py on GitHub.

Some References

For wiring diagrams, more background, and Arduino code for the CD4021, read Arduino ShiftIn. For the SN74LS165, read: Arduino: SN74HC165N, 74HC165 8 bit Parallel in/Serial out Shift Register, or Sparkfun: Shift Registers.

Of course, you can use a shift register for output as well as input. In that case you need a SIPO (Serial In, Parallel Out) shift register like a 74HC595. See Arduino ShiftOut: Serial to Parallel Shifting-Out with a 74HC595 Interfacing 74HC595 Serial Shift Register with Raspberry Pi. Another, less common option is the 74HC164N: Using a SN74HC164N Shift Register With Raspberry Pi

For input from my keyboard, initially I used three CD4021s. It basically worked, and you can see the code for it at keyboard.py (older version, for CD4021 shift registers), on GitHub.

But it turned out that looping over all those bits was slow -- I've been advised that you should wait at least 25 microseconds between bits for the CD4021, and even at 10 microseconds I found there wasa significant delay between hitting the key and hearing the note.I thought it might be all the fancy numpy code to generate waveforms for the chords, but when I used the Python profiler, it said most of the program's time was taken up in time.sleep(). Fortunately, there's a faster solution than shift registers: port expanders, which I'll talk about in Multiplexing Part 2: Port Expanders.

fwupd now tells you about known issues

After a week of being sick and not doing much, I’m showing the results of a day-or-so of hacking:

So, most of that being familiar to anyone that’s followed my previous blog posts. But wait, what’s that about a known issue?

That one little URL for the user to click on is the result of a rule engine being added to the LVFS. Of course, firmware updates shouldn’t ever fail, but in the real world they do, because distros don’t create /boot/efi correctly (cough, Arch Linux) or just because some people are running old versions of efivar, a broken git snapshot of libfwupdate or because a vendor firmware updater doesn’t work with secure boot turned on (urgh). Of all the failures logged on the LVFS, 95% fall into about 3 or 4 different failure causes, and if we know hundreds of people are hitting an issue we already understand we can provide them with some help.

So, how does this work? If you’re a user you don’t see any of this, you just download the metadata and firmware semi-automatically and get on with your life. If you’re a blessed hardware vendor on the LVFS (i.e. you can QA the updates into the stable branch) you can also create and view the rules for firmware owned by just your vendor group:

This new functionality will be deployed to the LVFS during the next downtime window. Comments welcome.

February 12, 2018

Shelved Wallpapers

GNOME 3.28 will release with another batch of new wallpapers that only a freaction of you will ever see. Apart from those I also made a few for different purposes that didn’t end up being used, but it would be a shame to keep shelved.

So here’s a bit of isometric goodness I quite enjoy on my desktop, you might as well.

Announcing Blender 2.8 Code Quest

Today the Blender Foundation announced the “Blender 2.8 Code Quest”, a crowd-funded event to gather the core developers to release the first version of the much anticipated Blender 2.8. It will be the first time that such a large development team will be working in one place. The crowd-funded goal is to get at least 10 contributors together for a period of 3 months, in the Blender Institute, Amsterdam, starting in April 2018.

For the Blender 2.8 project – with focus on workflow and an advanced new viewport system – several complex architectural changes are being added. The current team of core developers working on Blender 2.8 is distributed over multiple continents. Collaborating online from different places, in different timezones, using chat and emails, is not always efficient and is slowing down the process.

To prevent the 2.8 project to drag for another year – and to lose the exciting momentum – the Blender Foundation and the Blender Institute will combine their efforts to invite the core 2.8 developers to work in Amsterdam for 3 months. They will be working at the Blender Institute along with a group of artists who are using Blender 2.8 to produce a short film. This ensures we can tackle issues with enough attention, including time to provide feedback on UI design and usability. At least one of the permanent seats in the team is reserved for a designer to perform these tasks full-time.

The crowd-funding campaign is hosted on blender.org. The goal is to reach at least 1000 contributors who will pledge $39 and in return they will receive a limited edition Blender Rocket USB drive.

Confirmed participants are:
Ton Roosendaal, Sergey Sharybin, Dalai Felinto, Campbell Barton, Brecht van Lommel, Bastien Montagne, Clement Foucault, Joshua Leung, Sybren Stüvel and Pablo Vazquez. More people to be announced.

Read more about the Code Quest: https://www.blender.org/2-8/quest/

February 11, 2018

Razer doesn’t care about Linux

tl;dr: Don’t buy hardware from Razer and expect firmware updates to fix security problems on Linux.

Razer is a vendor that makes high-end gaming hardware, including laptops, keyboards and mice. I opened a ticket with Razor a few days ago asking them if they wanted to support the LVFS project by uploading firmware and sharing the firmware update protocol used. I offered to upstream any example code they could share under a free license, or to write the code from scratch given enough specifications to do so. This is something I’ve done for other vendors, and doesn’t take long as most vendor firmware updaters all do the same kind of thing; there are only so many ways to send a few kb of data to USB devices. The fwupd project provides high-level code for accessing USB devices, so yet-another-update-protocol is no big deal. I explained all about the LVFS, and the benefits it provided to a userbase that is normally happy to vote using their wallet to get hardware that’s supported on the OS of their choice.

I just received this note on the ticket, which was escalated appropriately:

I have discussed your offer with the dedicated team and we are thankful for your enthusiasm and for your good idea.
I am afraid I have also to let you know that at this moment in time our support for software is only focused on Windows and Mac.

The CEO of Razer Min-Liang Tan said recently “We’re inviting all Linux enthusiasts to weigh in at the new Linux Corner on Insider to post feedback, suggestions and ideas on how we can make it the best notebook in the world that supports Linux.” If this is true, and more than just a sound-bite, supporting the LVFS for firmware updates on the Razer Blade to solve security problems like Meltdown and Spectre ought to be a priority?

Certainly if peripheral updates or system firmware UpdateCapsule are not supportable on Linux, it would be good to correct well read articles as those makes it sound like Razor is interested in Linux users, of which the reality seems somewhat less optimistic. I’ve updated the vendor list with this information to avoid other people asking or filing tickets. Disappointing, but I’ll hopefully have some happier news soon about a different vendor.

February 05, 2018

security things in Linux v4.15

Previously: v4.14.

Linux kernel v4.15 was released last week, and there’s a bunch of security things I think are interesting:

Kernel Page Table Isolation
PTI has already gotten plenty of reporting, but to summarize, it is mainly to protect against CPU cache timing side-channel attacks that can expose kernel memory contents to userspace (CVE-2017-5754, the speculative execution “rogue data cache load” or “Meltdown” flaw).

Even for just x86_64 (as CONFIG_PAGE_TABLE_ISOLATION), this was a giant amount of work, and tons of people helped with it over several months. PowerPC also had mitigations land, and arm64 (as CONFIG_UNMAP_KERNEL_AT_EL0) will have PTI in v4.16 (though only the Cortex-A75 is vulnerable). For anyone with really old hardware, x86_32 is under development, too.

An additional benefit of the x86_64 PTI is that since there are now two copies of the page tables, the kernel-mode copy of the userspace mappings can be marked entirely non-executable, which means pre-SMEP hardware now gains SMEP emulation. Kernel exploits that try to jump into userspace memory to continue running malicious code are dead (even if the attacker manages to turn SMEP off first). With some more work, SMAP emulation could also be introduced (to stop even just reading malicious userspace memory), which would close the door on these common attack vectors. It’s worth noting that arm64 has had the equivalent (PAN emulation) since v4.10.

retpoline
In addition to the PTI work above, the retpoline kernel mitigations for CVE-2017-5715 (“branch target injection” or “Spectre variant 2”) started landing. (Note that to gain full retpoline support, you’ll need a patched compiler, as appearing in gcc 7.3/8+, and currently queued for release in clang.)

This work continues to evolve, and clean-ups are continuing into v4.16. Also in v4.16 we’ll start to see mitigations for the other speculative execution variant (i.e. CVE-2017-5753, “bounds check bypass” or “Spectre variant 1”).

x86 fast refcount_t overflow protection
In v4.13 the CONFIG_REFCOUNT_FULL code was added to stop many types of reference counting flaws (with a tiny performance loss). In v4.14 the infrastructure for a fast overflow-only refcount_t protection on x86 (based on grsecurity’s PAX_REFCOUNT) landed, but it was disabled at the last minute due to a bug that was finally fixed in v4.15. Since it was a tiny change, the fast refcount_t protection was backported and enabled for the Longterm maintenance kernel in v4.14.5. Conversions from atomic_t to refcount_t have also continued, and are now above 168, with a handful remaining.

%p hashing
One of the many sources of kernel information exposures has been the use of the %p format string specifier. The strings end up in all kinds of places (dmesg, /sys files, /proc files, etc), and usage is scattered through-out the kernel, which had made it a very hard exposure to fix. Earlier efforts like kptr_restrict‘s %pK didn’t really work since it was opt-in. While a few recent attempts (by William C Roberts, Greg KH, and others) had been made to provide toggles for %p to act like %pK, Linus finally stepped in and declared that %p should be used so rarely that it shouldn’t used at all, and Tobin Harding took on the task of finding the right path forward, which resulted in %p output getting hashed with a per-boot secret. The result is that simple debugging continues to work (two reports of the same hash value can confirm the same address without saying what the address actually is) but frustrates attacker’s ability to use such information exposures as building blocks for exploits.

For developers needing an unhashed %p, %px was introduced but, as Linus cautioned, either your %p remains useful when hashed, your %p was never actually useful to begin with and should be removed, or you need to strongly justify using %px with sane permissions.

It remains to be seen if we’ve just kicked the information exposure can down the road and in 5 years we’ll be fighting with %px and %lx, but hopefully the attitudes about such exposures will have changed enough to better guide developers and their code.

struct timer_list refactoring
The kernel’s timer (struct timer_list) infrastructure is, unsurprisingly, used to create callbacks that execute after a certain amount of time. They are one of the more fundamental pieces of the kernel, and as such have existed for a very long time, with over 1000 call sites. Improvements to the API have been made over time, but old ways of doing things have stuck around. Modern callbacks in the kernel take an argument pointing to the structure associated with the callback, so that a callback has context for which instance of the callback has been triggered. The timer callbacks didn’t, and took an unsigned long that was cast back to whatever arbitrary context the code setting up the timer wanted to associate with the callback, and this variable was stored in struct timer_list along with the function pointer for the callback. This creates an opportunity for an attacker looking to exploit a memory corruption vulnerability (e.g. heap overflow), where they’re able to overwrite not only the function pointer, but also the argument, as stored in memory. This elevates the attack into a weak ROP, and has been used as the basis for disabling SMEP in modern exploits (see retire_blk_timer). To remove this weakness in the kernel’s design, I refactored the timer callback API and and all its callers, for a whopping:

1128 files changed, 4834 insertions(+), 5926 deletions(-)

Another benefit of the refactoring is that once the kernel starts getting built by compilers with Control Flow Integrity support, timer callbacks won’t be lumped together with all the other functions that take a single unsigned long argument. (In other words, some CFI implementations wouldn’t have caught the kind of attack described above since the attacker’s target function still matched its original prototype.)

That’s it for now; please let me know if I missed anything. The v4.16 merge window is now open!

© 2018, Kees Cook. This work is licensed under a Creative Commons Attribution-ShareAlike 3.0 License.
Creative Commons License

Interview with Owly Owlet

 

Could you tell us something about yourself?

Hello. I’m Maria, more often I use my nickname: Owly Owlet. I have a youtube channel, where I make video tutorials (in Russian) about how to use art software, mostly Krita.

Do you paint professionally, as a hobby artist, or both?

Art is my hobby, but I wish I could become a professional artist someday. For now there is much to be learned.

What genre(s) do you work in?

My art usually is more cartoony-like. I like fantasy world, fairy tales with medieval clothes, castles and magical creatures.

Whose work inspires you most — who are your role models as an artist?

There are so many incredible artists, whose art makes me want to learn and practice drawing more and more, it’s immensely hard to pick just a single one. But for now I really found of Andreas Rocha’s work. I also love the art style of David Revoy.

How and when did you get to try digital painting for the first time?

I’ve been drawing digitally since March 2017, so almost a year now. My husband gave me a tablet as a present for my birthday. Before that I drew with vector tools and a mouse a bit.

What makes you choose digital over traditional painting?

The freedom you have with digital art. With traditional painting you have to learn not only the basics of how to draw: perspective, light, color… You also have to learn how to work with different tools: pencils, markers, watercolor, acrylic paint, oil paint and so on. And it’s harder to fix your mistakes when you are just learning. And when you draw digitally, you have the magic of “Ctrl+Z” and layers. And besides that you can change the color scheme, mirror your image which makes way easier to identify and fix your proportion and color mistakes. You just have to find the software you are comfortable to paint with and you are good to go.

How did you find out about Krita?

It was Age of Asparagus’ “Krita meets Bob Ross” tutorials. They are awesome and really helped me to learn how to use Krita and not to be afraid of it. (https://www.youtube.com/channel/UCkKFLSJjYtKNdFy3P7Q-CAA)

What was your first impression?

Before Krita I used FireAlpaca a bit, which is fine software too, especially for a beginner. So, switching to Krita after FireAlpaca was a bit scary, you know that “so many buttons” kind of thing.

What do you love about Krita?

My favourite is the Assistant Tool, vanishing point and perspective. And I also love the dynamic brush and the quality of mixing brushes.

What do you think needs improvement in Krita? Is there anything that really annoys you?

That would be nice if Krita had not only Brightness/Contrast curve, but also contrast sliders, like those in Photoshop.

What sets Krita apart from the other tools that you use?

It has so many cool features and tools, so flexible when it comes to brush settings and working with color, and yet it’s free. Isn’t that amazing?

If you had to pick one favourite of all your work done in Krita so far, what would it be, and why?


I think this was the first one decent enough. When I drew it, I thought that I was finally getting somewhere.

What techniques and brushes did you use in it?

Most of the time for cartoony characters I use basic Krita brushes, just edit settings a bit. Airbrush_pressure for sketching and shading, Fill_circle for colouring, ink_ballpen for lineart, Basic_wet_soft for blending.

Where can people see more of your work?

My drawings on Deviantart: https://owlyowlet.deviantart.com/
My Krita tutorials (in Russian): https://www.youtube.com/c/СтудияСовятня

Anything else you’d like to share?

I wanted to say that I admire people who created Krita, who work on it, the developers, artists, translators, test engineers and other good people who make it possible for everyone to learn digital art with Krita. Thank you for all your hard work.

February 04, 2018

FreeCAD Arch development news - January 2018

Hi everybody, Sorry about the late news again, but it would have been a pity not to include a fresh feedback from the FOSDEM that happened this weekend. So here are the main topics that happened in the last month. Once again, thanks a million to eveybody who helps the effort by contributing to my Patreon...

February 02, 2018

Raspberry Pi Console over USB: Configuring an Ethernet Gadget

When I work with a Raspberry Pi from anywhere other than home, I want to make sure I can do what I need to do without a network.

With a Pi model B, you can use an ethernet cable. But that doesn't work with a Pi Zero, at least not without an adapter. The lowest common denominator is a serial cable, and I always recommend that people working with headless Pis get one of these; but there are a lot of things that are difficult or impossible over a serial cable, like file transfer, X forwarding, and running any sort of browser or other network-aware application on the Pi.

Recently I learned how to configure a Pi Zero as a USB ethernet gadget, which lets you network between the Pi and your laptop using only a USB cable. It requires a bit of setup, but it's definitely worth it. (This apparently only works with Zero and Zero W, not with a Pi 3.)

The Cable

The first step is getting the cable. For a Pi Zero or Zero W, you can use a standard micro-USB cable: you probably have a bunch of them for charging phones (if you're not an Apple person) and other devices.

Set up the Pi

Setting up the Raspberry Pi end requires editing two files in /boot, which you can do either on the Pi itself, or by mounting the first SD card partition on another machine.

In /boot/config.txt add this at the end:

dtoverlay=dwc2

In /boot/cmdline.txt, at the end of the long list of options but on the same line, add a space, followed by: modules-load=dwc2,g_ether

Set a static IP address

This step is optional. In theory you're supposed to use some kind of .local address that Bonjour (the Apple protocol that used to be called zeroconf, and before that was called Rendezvous, and on Linux machines is called Avahi). That doesn't work on my Linux machine. If you don't use Bonjour, finding the Pi over the ethernet link will be much easier if you set it up to use a static IP address. And since there will be nobody else on your USB network besides the Pi and the computer on the other end of the cable, there's no reason not to have a static address: you're not going to collide with anybody else.

You could configure a static IP in /etc/network/interfaces, but that interferes with the way Raspbian handles wi-fi via wpa_supplicant and dhcpcd; so you'd have USB networking but your wi-fi won't work any more.

Instead, configure your address in Raspbian via dhcpcd. Edit /etc/dhcpcd.conf and add this:

interface usb0
static ip_address=192.168.7.2
static routers=192.168.7.1
static domain_name_servers=192.168.7.1

This will tell Raspbian to use address 192.168.7.2 for its USB interface. You'll set up your other computer to use 192.168.7.1.

Now your Pi should be ready to boot with USB networking enabled. Plug in a USB cable (if it's a model A or B) or a micro USB cable (if it's a Zero), plug the other end into your computer, then power up the Pi.

Setting up a Linux machine for USB networking

The final step is to configure your local computer's USB ethernet to use 192.168.7.1.

On Linux, find the name of the USB ethernet interface. This will only show up after you've booted the Pi with the ethernet cable plugged in to both machines.

ip a
The USB interface will probably start eith en and will probably be the last interface shown.

On my Debian machine, the USB network showed up as enp0s26u1u1. So I can configure it thusly (as root, of course):

ip a add 192.168.7.1/24 dev enp0s26u1u1
ip link set dev enp0s26u1u1 up
(You can also use the older ifconfig rather than ip: sudo ifconfig enp0s26u1u1 192.168.7.1 up)

You should now be able to ssh into your Raspberry Pi using the address 192.168.7.2, and you can make an appropriate entry in /etc/hosts, if you wish.

For a less hands-on solution, if you're using Mac or Windows, try Adafruit's USB gadget tutorial. It's possible that might also work for Linux machines running Avahi. If you're using Windows, you might prefer CircuitBasics' ethernet gadget tutorial.

Happy networking!

February 01, 2018

Firmware Telemetry for Vendors

We’ve shipped nearly 1.2 MILLION firmware updates out to Linux users since we started the LVFS project.

I found out this nugget of information using a new LVFS vendor feature, soon to be deployed: Telemetry. This builds on the previously discussed success/failure reporting and adds a single page for the vendor to get statistics about each bit of hardware. Until more people are running the latest fwupd and volunteering to share their update history it’s less useful, but still interesting until then.

No new batches of ColorHug2

I was informed by AMS (the manufacturer that makes the XYZ sensor that’s the core of the CH2 device) that the AS73210 (aka MTCSiCF) and the MTI08D are end of life products. The replacement for the sensor the vendor offers is the AS73211, which of course is more expensive and electrically incompatible with the AS73210.

The somewhat-related new AS7261 sensor does look interesting as it somewhat crosses the void between a colorimeter and something that can take non-emissive readings, but it’s a completely different sensor to the one on the ColorHug2, and mechanically to the now-abandoned ColorHug+. I’m also feeling twice burned buying specialist components from single-source suppliers.

Being a parents to a 16 week old baby doesn’t put Ania and I in a position where I can go through the various phases of testing, prototypes, test batch, production batch etc for a device refresh like we did with the ColorHug -> ColorHug2. I’m hoping I can get a chance to play with some more kinds of sensors from different vendors, although that’s not going to happen before I start getting my free time back. At the moment I have about 50 fully completed ColorHug2 devices in boxes ready to be sold.

In the true spirit of OpenHardware and free enterprise, if anyone does want to help with the design of a new ColorHug device I’m open for ideas. ColorHug was really just a hobby that got out of control, and I’d love for someone else to have the thrill and excitement of building a nano-company from scratch. Taking me out the equation completely, I’d be as equally happy referring on people who want to buy a ColorHug upgrade or replacement to a different project, if the new product met with my approval :)

So, 50 ColorHugs should last about 3 months before stock runs out, but I know a few people are using devices on production lines and other sorts of industrial control — if that sounds familiar, and you’d like to buy a spare device, now is the time to do so. Of course, I’ll continue supporting all the existing 3162 devices well into the future. I hope to be back building OpenHardware soon, and hopefully with a new and improved ColorHug3.

January 30, 2018

Another Morevna fundraiser!

Nikolai Mamashev, of Pepper & Carrot animation fame, and his team are working on a new episode of the Morevna open-source animation series. As in the previous project, they’re using Krita for creating and processing the artwork and Blender and other open-source tools for animation. Everything will be published under the Creative Commons Attribution-ShareAlike license.

Nikolai and his team are running a crowdfunding campaign to make this happen. Among the rewards:

– a training video course so you, too can master the techniques used to create this animation

– your own image animated! Check out this one of Kiki from the previous campaign:

And many other exclusive rewards! By claiming any of them, you are helping Nikolai and the team of Morevna to bring one more animation project powered by open-source technologies. The success of this project depends on your support!

The series is a futuristic adaptation of the traditional Russian fairy tale “Marya Morevna” (also known as “The Death of Koschei the Deathless”). The action takes place in the distant future, where horses are replaced by bikes and cars, the main protagonist Ivan Tsarevich turns into a talented mechanic, Marya Morevna is a biker queen with a samurai sword, and Koschei is more immortal and evil than ever – as he is a battle robot now!


Support the new episode of Morevna

January 26, 2018

Go: debugging multiple response.WriteHeader calls

Say you’re building a HTTP service in Go and suddenly it starts giving you these:

http: multiple response.WriteHeader calls

Horrible when that happens, right?

It’s not always very easy to figure out why you get them and where they come from. Here’s a hack to help you trace them back to their origin:

type debugLogger struct{}

func (d debugLogger) Write(p []byte) (n int, err error) {
	s := string(p)
	if strings.Contains(s, "multiple response.WriteHeader") {
		debug.PrintStack()
	}
	return os.Stderr.Write(p)
}

// Now use the logger with your http.Server:
logger := log.New(debugLogger{}, "", 0)

server := &http.Server{
    Addr:     ":3001",
    Handler:  s,
    ErrorLog: logger,
}
log.Fatal(server.ListenAndServe())

This will output a nice stack trace whenever it happens. Happy hacking!


Comments | More on rocketeer.be | @rubenv on Twitter

January 25, 2018

Tricks for Installing a Laser Printer on Linux in CUPS

(Wherein I rant about how bad CUPS has become.)

I had to set up two new printers recently. CUPS hasn't gotten any better since the last time I bought a printer, maybe five years ago; in fact, it's gotten quite a bit worse. I'm amazed at how difficult it was to add these fairly standard laser printers, both of which I'd researched beforehand to make sure they worked with Linux.

It took me about three hours for the first printer. The second one, a few weeks later, "only" took about 45 minutes ... at which point I realized I'd better write everything down so it'll be faster if I need to do it again, or if I get the silly notion that I might want to print from another computer, like my laptop.

I used the CUPS web interface; I didn't try any of the command-line tools.

Figure out the connection type

In the CUPS web interface, after you log in and click on Administration, whether you click on Find New Printers or Add Printer, you're faced with a bunch of identical options with no clue how to choose between them. For example, Find New Printers with a Dell E310dw connected shows:

Available Printers
  • [Add This Printer] Virtual Braille BRF Printer (CUPS-BRF)
  • [Add This Printer] Dell Printer E310dw (Dell Printer E310dw)
  • [Add This Printer] Dell Printer E310dw (Dell Printer E310dw)
  • [Add This Printer] Dell Printer E310dw (Dell Printer E310dw (driverless))

What is a normal human supposed to do with this? What's the difference between the three E210dw entries and which one am I supposed to choose? (Skipping ahead: None of them.) And why is it finding a virtual Braille BRF Printer?

The only way to find out the difference is to choose one, click on Next and look carefully at the URL. For the three E310dw options above, that gives:

  • dnssd://Dell%20Printer%20E310dw._ipp._tcp.local/?uuid=[long uuid here]
  • lpd://DELL316BAA/BINARY_P1
  • ipp://DELL316BAA.local:631/ipp/print

Again skipping ahead: none of those are actually right. Go ahead, try all three of them and see. You'll get error messages about empty PPD files. But while you're trying them, write down, for each one, the URL listed as Connection (something like the dnssd:, lpd: or ipp: URLs listed above); and note, in the driver list after you click on your manufacturer, how many entries there are for your printer model, and where they show up in the list. You'll need that information later.

Download some drivers

Muttering about the idiocy of all this -- why ship empty drivers that won't install? Why not just omit drivers if they're not available? Why use the exact same name for three different printer entries and four different driver entries? -- the next step is to download and install the manufacturer's drivers. If you're on anything but Redhat, you'll probably either need to download an RPM and unpack it, or else google for the hidden .deb files that exist on both Dell's and Brother's websites that their sites won't actually find for you.

It might seem like you could just grab the PPD from inside those RPM files and put it wherever CUPS is finding empty ones, but I never got that to work. Much as I dislike installing proprietary .deb files, for both printers that was the only method I found that worked. Both Dell and Brother have two different packages to install. Why two and what's the difference? I don't know.

Once you've installed the printer driver packages, you can go back to the CUPS Add Printer screen. Which hasn't gotten any clearer than before. But for both the Brother and the Dell, ipp: is the only printer protocol that worked. So try each entry until you find the one that starts with ipp:.

Set up an IP address and the correct URL

But wait, you're not done. Because CUPS gives you a URL like ipp://DELL316BAA.local:631/ipp/print, and whatever that .local thing is, it doesn't work. You'll be able to install the printer, but when you try to print to it it fails with "unable to locate printer".

(.local apparently has something to do with assuming you're running a daemon that does "Bonjour", the latest name for the Apple service discovery protocol that was originally called Rendezvous, then renamed to Zeroconf, then to Bonjour. On Linux it's called Avahi, but even with an Avahi daemon this .local thing didn't work for me. At least it made me realize that I had the useless Avahi daemon running, so now I can remove it.).

So go back to Add Printer and click on Internet Printing Protocol (ipp) under Other network printers and click Continue. That takes you to a screen that suggests that you want URLs like:

http://hostname:631/ipp/
http://hostname:631/ipp/port1

ipp://hostname/ipp/
ipp://hostname/ipp/port1

lpd://hostname/queue

socket://hostname
socket://hostname:9100

None of these is actually right. What these printers want -- at least, what both the Brother and the Dell wanted -- was ipp://printerhostname:631/ipp/print

printerhostname? Oh, did I forget to mention static IP? I definitely recommend that you make a static IP for your printer, or at least add it to your router's DHCP list so it always gets the same address. Then you can make an entry in /etc/hosts for printerhostname. I guess that .local thing was supposed to compensate for an address that changes all the time, which might be a nifty idea if it worked, but since it doesn't, make a static IP and use it in your ipp: URL.

Choose a driver

Now, finally! you can move on to choosing a driver. After you pick the manufacturer, you'll be presented with a list that probably includes at least three entries for your printer model. Here's where it helps if you paid attention to how the list looked before you installed the manufacturer's drivers: if there's a new entry for your printer that wasn't there before, that's the non-empty one you want. If there are two or more new entries for your printer that weren't there before, as there were for the Dell ... shrug, all you can do is pick one and hope.

Of course, once you manage to get through configuration to "Printer successfully added", you should immediately run Maintenance->Print Test Page. You may have to power cycle the printer first since it has probably gone to sleep while you were fighting with CUPS.

All this took me maybe three hours the first time, but it only took me about 45 minutes the second time. Hopefully now that I've written this, it'll be much faster next time. At least if I don't succumb to the siren song of thinking a fairly standard laser printer ought to have a driver that's already in CUPS, like they did a decade ago, instead of always needing a download from the manufacturer.

If laser printers are this hard I don't even want to think about what it's like to install a photo printer on Linux these days.

January 23, 2018

GCab and CVE-2018-5345

tl;dr: Update GCab from your distributor.

Longer version: Just before Christmas I found a likely exploitable bug in the libgcab library. Various security teams have been busy with slightly more important issues, and so it’s taken a lot longer than usual to be verified and assigned a CVE. The issue I found was that libgcab attempted to read a large chunk into a small buffer, overwriting lots of interesting things past the end of the buffer. ALSR and SELinux saves us in nearly all cases, so it’s not the end of the world. Almost a textbook C buffer overflow (rust, yada, whatever) so it was easy to fix.

Some key points:

  • This only affects libgcab, not cabarchive or libarchive
  • All gcab versions less than 0.8 are affected
  • Anything that links to gcab is affected, so gnome-software, appstream-glib and fwupd at least
  • Once you install the fixed gcab you need to restart anything that’s using it, e.g. fwupd
  • There is no silly branded name for this bug
  • The GCab project is incredibly well written, and I’ve been hugely impressed with the code quality
  • You can test if your GCab has been fixed by attempting to decompress this file, if the program crashes, you need to update

With Marc-André’s blessing, I’ve released version v0.8 of gcab with this fix. I’ve also released v1.0 which has this fix (and many more nice API additions) which also switches the build system to Meson and cleans up a lot of leaks using g_autoptr(). If you’re choosing a version to update to, the answer is probably 1.0 unless you’re building for something more sedate like RHEL 5 or 6. You can get the Fedora 27 packages here or they’ll be on the mirrors tomorrow.

January 22, 2018

darktable 2.4.1 released

we’re proud to announce the first bugfix release for the 2.4 series of darktable, 2.4.1!

the github release is here: https://github.com/darktable-org/darktable/releases/tag/release-2.4.1.

as always, please don’t use the autogenerated tarball provided by github, but only our tar.xz. the checksums are:

$ sha256sum darktable-2.4.1.tar.xz
6254c63f9b50894b3fbf431d98c0fe8ec481957ab91f9af76e33cc1201c29704 darktable-2.4.1.tar.xz
$ sha256sum darktable-2.4.1.dmg
75077f17332a6fda144125ab0f1d3dd219c214bf7602b0b252208f1ec665d031 darktable-2.4.1.dmg
$ sha256sum darktable-2.4.1-win64.exe
0be1e0dd8dec61a7cea41598c52db258edaee8783c543b4311fa0ac56ab43d2a darktable-2.4.1-win64.exe
$ sha256sum darktable-2.4.1-win64.zip
560d82e4c87c002f0284daca922023df136c822713e3670ba42358c9427fe26c darktable-2.4.1-win64.zip

when updating from the currently stable 2.2.x series, please bear in mind that your edits will be preserved during this process, but it will not be possible to downgrade from 2.4 to 2.2.x any more.

Important note: to make sure that darktable can keep on supporting the raw file format for your camera, please read this post on how/what raw samples you can contribute to ensure that we have the full raw sample set for your camera under CC0 license!

and the changelog as compared to 2.4.0 can be found below.

New Features

  • Allow to select the GUI language in the preferences
  • Add a filter rule to the collect module to find locally copied images
  • Add favourite toggle to darkroom modules’ right click popup
  • Allow blending/masking in the hot pixels module
  • Add keyboard shortcuts to zoom and pan an image in darkroom. Panning uses the arrow keys, zooming defaults to ctrl- and ctrl+. Use alt and ctrl to change the step size of panning.
  • Some minor speedups in the grain module
  • Handling stdout on Windows: do not redirect stdout for simple command line arguments (--help and --version)
  • On Windows, show the location of the log file in the help message
  • Enable searching in the more modules list – click into the list to give focus to it, then start typing. The default GTK shortcut ctrl-f doesn’t work as it’s used for filmstrip already
  • Add a debug print when compiling OpenCL kernels

Bugfixes

  • Use the configured overwrite color profile when exporting from Lua – this broke GIMP integration
  • Support presets with < in their name
  • Fix export to non-existing path with \ as the path separator on Windows
  • Don’t insist on the db being locked when it doesn’t even exist
  • Don’t touch the mix slider when resetting the curve in color zones
  • Fix a bug in the exposure module that would only allow corrections of up to 10 stops
  • Fix custom shortcuts with shift modifier
  • Properly ellipsize text in the recently used collections list
  • Fix exported galeries with filenames containing a '
  • Fix finding mipmaps cache folder in purge_from_cache.sh script
  • Fix a crash in the recently used collections list due to a broken config file
  • Set the sqlite threading mode to Serialized
  • Fix old export presets using OpenEXR
  • Fix building with clang on Windows

Changed Dependencies

  • iso-codes version 3.66 or newer is suggested for a nicer list of translations in the preferences.
  • The Windows installer comes with an updated libexiv2 so TIFF exports should be much faster now

Camera support, compared to 2.4.0

Warning: support for Nikon NEF ‘lossy after split’ raws was unintentionally broken due to the lack of such samples. Please see this post for more details. If you have affected raws, please contribute samples!

Base Support

  • Panasonic DC-G9 (4:3)
  • Paralenz Dive Camera (chdk)
  • Pentax KP
  • Sjcam SJ6 LEGEND (chdk-b, chdk-c)

White Balance Presets

  • Leaf Credo 40
  • Nikon D3400
  • Olympus E-M1MarkII
  • Panasonic DC-G9
  • Sony ILCE-7RM3

Noise Profiles

  • Canon EOS 750D
  • Canon EOS Kiss X8i
  • Canon EOS Rebel T6i
  • Canon EOS 77D
  • Canon EOS 9000D
  • Canon EOS M100
  • Canon EOS M6
  • Sony DSC-RX100M4
  • YI TECHNOLOGY M1

Translations

  • Czech
  • Dutch
  • French
  • German
  • Hebrew
  • Hungarian
  • Italian
  • Slovenian

Interview with Baukje Jagersma

Source of infinity

Could you tell us something about yourself?

Hey! My name Is Baukje Jagersma, I’m 22 years old and live in the Netherlands. I studied game design and recently started doing freelance, to try and make a living out of something I enjoy doing!

Do you paint professionally, as a hobby artist, or both?

Both: I’ve always enjoyed creating my own stories and worlds with drawing and recently started doing freelance work as well.

What genre(s) do you work in?

Most if not all of my work has something to do with fantasy. To me that’s the best part of drawing, you can create things that don’t exist and make them look believable. Besides that I mostly work as an illustrator and concept artist.

Whose work inspires you most — who are your role models as an artist?

There are a lot of sources where I get inspiration from, art in games for example, movies or art sites.

A few artists that are really worth mentioning would be Grzegorz Rutkowski, Ruan Jia and Piotr Jablonski.

How and when did you get to try digital painting for the first time?

Probably when I first discovered Deviantart. I was already familiar with GIMP, which I used to create photo-manipulations with. But seeing all the amazingly talented artists on there made me want to try out digital painting for myself.

What makes you choose digital over traditional painting?

I feel like traditional has more limitations and can get messy. In digital you can easily pick any color you like, or undo something that doesn’t work. For me it just works a lot faster.

Forest elf and her kitty

How did you find out about Krita?

Somewhere around 2013-2014 when an artist posted his Krita art on a GIMP forum.

What was your first impression?

I really didn’t know where to start, haha! There were just so many more options than I was used to in GIMP, especially with all the individual brush engines. It really took me a while to get comfortable with the program.

What do you love about Krita?

Now I’ve just grown to love the multiple brush engines! The wrap-around mode, animation tool, brush smoothing options, symmetry options, assistant tool and the different layer and mask options are probably the key features that I love about it. It’s a program that just has so much to offer which makes it a lot of fun to explore with!

What do you think needs improvement in Krita? Is there anything that really annoys you?

Probably the only thing that really bugs me is the text tool, which seems to have a few weird issues right now. I’d also love to see the possibility to import and use vector layers and an alternative to the pattern brush option to make it work less repetitive (something similar to Photoshop’s dual brush perhaps).

What sets Krita apart from the other tools that you use?

Kinda mentioned it earlier already, it has a lot to offer which makes it fun to explore with! Besides that it’s available to everyone and works just as well as any other ‘professional’ digital painting program.

If you had to pick one favourite of all your work done in Krita so far, what would it be, and why?

Probably one of my few non-illustrative works. I really wanted to try out the animation tool so I decided to try out a run cycle. I had little knowledge of animating beforehand- but I like how the animation and design turned out in the end.

Tiger run cycle animation

What techniques and brushes did you use in it?

I made a few different style concepts beforehand, where I chose a design from and later on used as a reference. I first made a sketch version of the animation which I then refined and colored. I actually made a little video about it which I posted on youtube.

Where can people see more of your work?

Deviantart: https://baukjespirit.deviantart.com/
Artstation: https://www.artstation.com/baukjespirit
Instagram: https://www.instagram.com/baukjespirit/
Twitter: https://twitter.com/BaukjeJagersma
Youtube: https://www.youtube.com/user/baukjespirit

Anything else you’d like to share?

I’d like to thank the Krita team for developing this amazing program and making it available to everyone! I’m very excited to see how Krita will further develop in the future!

January 21, 2018

Reading Buttons from a Raspberry Pi

When you attach hardware buttons to a Raspberry Pi's GPIO pin, reading the button's value at any given instant is easy with GPIO.input(). But what if you want to watch for button changes? And how do you do that from a GUI program where the main loop is buried in some library?

Here are some examples of ways to read buttons from a Pi. For this example, I have one side of my button wired to the Raspberry Pi's GPIO 18 and the other side wired to the Pi's 3.3v pin. I'll use the Pi's internal pulldown resistor rather than adding external resistors.

The simplest way: Polling

The obvious way to monitor a button is in a loop, checking the button's value each time:

import RPi.GPIO as GPIO
import time

button_pin = 18

GPIO.setmode(GPIO.BCM)

GPIO.setup(button_pin, GPIO.IN, pull_up_down = GPIO.PUD_DOWN)

try:
    while True:
        if GPIO.input(button_pin):
            print("ON")
        else:
            print("OFF")

        time.sleep(1)

except KeyboardInterrupt:
    print("Cleaning up")
    GPIO.cleanup()

But if you want to be doing something else while you're waiting, instead of just sleeping for a second, it's better to use edge detection.

Edge Detection

GPIO.add_event_detect, will call you back whenever it sees the pin's value change. I'll define a button_handler function that prints out the value of the pin whenever it gets called:

import RPi.GPIO as GPIO
import time

def button_handler(pin):
    print("pin %s's value is %s" % (pin, GPIO.input(pin)))

if __name__ == '__main__':
    button_pin = 18

    GPIO.setmode(GPIO.BCM)

    GPIO.setup(button_pin, GPIO.IN, pull_up_down = GPIO.PUD_DOWN)

    # events can be GPIO.RISING, GPIO.FALLING, or GPIO.BOTH
    GPIO.add_event_detect(button_pin, GPIO.BOTH,
                          callback=button_handler,
                          bouncetime=300)

    try:
        time.sleep(1000)
    except KeyboardInterrupt:
        GPIO.cleanup()

Pretty nifty. But if you try it, you'll probably find that sometimes the value is wrong. You release the switch but it says the value is 1 rather than 0. What's up?

Debounce and Delays

The problem seems to be in the way RPi.GPIO handles that bouncetime=300 parameter.

The bouncetime is there because hardware switches are noisy. As you move the switch from ON to OFF, it doesn't go cleanly all at once from 3.3 volts to 0 volts. Most switches will flicker back and forth between the two values before settling down. To see bounce in action, try the program above without the bouncetime=300. There are ways of fixing bounce in hardware, by adding a capacitor or a Schmitt trigger to the circuit; or you can "debounce" the button in software, by waiting a while after you see a change before acting on it. That's what the bouncetime parameter is for.

But apparently RPi.GPIO, when it handles bouncetime, doesn't always wait quite long enough before calling its event function. It sometimes calls button_handler while the switch is still bouncing, and the value you read might be the wrong one. Increasing bouncetime doesn't help. This seems to be a bug in the RPi.GPIO library.

You'll get more reliable results if you wait a little while before reading the pin's value:

def button_handler(pin):
    time.sleep(.01)    # Wait a while for the pin to settle
    print("pin %s's value is %s" % (pin, GPIO.input(pin)))

Why .01 seconds? Because when I tried it, .001 wasn't enough, and if I used the full bounce time, .3 seconds (corresponding to 300 millisecond bouncetime), I found that the button handler sometimes got called multiple times with the wrong value. I wish I had a better answer for the right amount of time to wait.

Incidentally, the choice of 300 milliseconds for bouncetime is arbitrary and the best value depends on the circuit. You can play around with different values (after commenting out the .01-second sleep) and see how they work with your own circuit and switch.

You might think you could solve the problem by using two handlers:

    GPIO.add_event_detect(button_pin, GPIO.RISING, callback=button_on,
                          bouncetime=bouncetime)
    GPIO.add_event_detect(button_pin, GPIO.FALLING, callback=button_off,
                          bouncetime=bouncetime)
but that apparently isn't allowed: RuntimeError: Conflicting edge detection already enabled for this GPIO channel.

Even if you look just for GPIO.RISING, you'll still get some bogus calls, because there are both rising and falling edges as the switch bounces. Detecting GPIO.BOTH, waiting a short time and checking the pin's value is the only reliable method I've found.

Edge Detection from a GUI Program

And now, the main inspiration for all of this: when you're running a program with a graphical user interface, you don't have control over the event loop. Fortunately, edge detection works fine from a GUI program. For instance, here's a simple TkInter program that monitors a button and shows its state.

import Tkinter
from RPi import GPIO
import time

class ButtonWindow:
    def __init__(self, button_pin):
        self.tkroot = Tkinter.Tk()
        self.tkroot.geometry("100x60")

        self.label = Tkinter.Label(self.tkroot, text="????",
                                   bg="black", fg="white")
        self.label.pack(padx=5, pady=10, side=Tkinter.LEFT)

        self.button_pin = button_pin
        GPIO.setmode(GPIO.BCM)

        GPIO.setup(self.button_pin, GPIO.IN, pull_up_down=GPIO.PUD_DOWN)

        GPIO.add_event_detect(self.button_pin, GPIO.BOTH,
                              callback=self.button_handler,
                              bouncetime=300)

    def button_handler(self, channel):
        time.sleep(.01)
        if GPIO.input(channel):
            self.label.config(text="ON")
            self.label.configure(bg="red")
        else:
            self.label.config(text="OFF")
            self.label.configure(bg="blue")

if __name__ == '__main__':
    win = ButtonWindow(18)
    win.tkroot.mainloop()

You can see slightly longer versions of these programs in my GitHub Pi Zero Book repository.

January 12, 2018

New Stable Release: Krita 3.3.3

Today we’re releasing Krita 3.3.3. This will probably be the last stable release in the Krita 3 series. This release contains several bug fixes and one very important change for Windows users:

  • The Direct3d/Angle renderer is now the default for Intel users. Recent updates to the Intel display drivers have broken Krita on many more systems than before, so it’s better that everyone gets this workaround by default. If you experience decreased performance, you can always try to enable OpenGL again.

Other fixes and improvements include:

  • Fix an issue where it would not be possible to select certain blending modes when the current layer is grayscale but the image is rgb.
  • Set the OS and platform when reporting a bug from within Krita on Windows.
  • Make it possible to enter color values as percentage in the specific color selector
  • Add OpenGL warnings and make ANGLE default on Intel GPUs
  • Add an Invert button to the levels filter
  • Implement loading and saving of styles for group layers to and from PSD
  • Fix the erase mode not showing correctly when returning to the brush tool
  • Save the visibility of individual assistants in .kra files
  • Add an option to draw ruler tips as a power of 2
  • Disable autoscroll on move and transform tools.
  • Improve handling of native mouse events when using a pen and the Windows Ink API
  • Fix the focal point for the pinch zoom gesture
  • Fix loading netpbm files with comment.

Download

Windows

Note for Windows users: if you encounter crashes, please follow these instructions to use the debug symbols so we can figure out where Krita crashes.

Linux

(If, for some reason, Firefox thinks it needs to load this as text: to download, right-click on the link.)

When it is updated, you can also use the Krita Lime PPA to install Krita 3.3.3 on Ubuntu and derivatives. We are working on an updated snap.

OSX

Note: the gmic-qt and pdf plugins are not available on OSX.

Source code

md5sums

For all downloads:

Key

The Linux appimage and the source tarball are signed. You can retrieve the public key over https here:
0x58b9596c722ea3bd.asc
. The signatures are here.

Support Krita

Krita is a free and open source project. Please consider supporting the project with donations or by buying training videos or the artbook! With your support, we can keep the core team working on Krita full-time.

January 11, 2018

Krita 4.0 Beta 1

We’ve officially gone into String Freeze mode now! That’s developer speak for “No New Features, Honest”. Everything that’s going into Krita 4.0 now is in, and the only thing left to do is fixing bugs and refining stuff.

Given how much has changed between Krita 3 and Krita 4, that’s an important part of the job! Let us here repeat a very serious warning.

THE FILE FORMAT FOR VECTOR LAYERS HAS CHANGED. IF YOU SAVE AN IMAGE WITH KRITA 4.0 THAT HAS VECTOR LAYERS, KRITA 3 CANNOT OPEN IT. IF YOU OPEN A KRITA 3 FILE WITH VECTOR LAYERS IN KRITA 4, THE VECTOR LAYERS MIGHT GET MESSED UP. BEFORE WORKING ON SUCH FILES IN KRITA 4, MAKE A BACKUP.

This doubles for files that contain text. Text in krita 3 is based on the ODT standard, text in Krita 4 is implemented using SVG. This beta is the first release that contains the new text tool. Here’s the low-down on the new text tool:

  • It’s not the text tool we wanted to create, and you could consider it as a stop-gap measure. Because of all the tax office worries, we simply didn’t have the time to create the fully-capable opentype-integrated text tool that we wanted to do.
  • But we couldn’t keep the old text tools either: they were broken, and based on ODT. We needed to have something that could replace those tools and that would would be functional enough for the simplest of use-cases, like text balloons in a comic.
  • So, what we’ve got is simple. There’s no vertical text for Chinese or Japanese yet, there’s no OpenType tweaking, there’s no fine typographic control.
  • The user interface is not final yet, and there are quite a few things that need polishing and fixing, and that we’re working on, but Krita 4’s text tool will be mostly what we’ve got now.

Apart from that…

This beta contains pretty much everything… We started working on some of these features, like the export feedback in 2016. Here’s a short list:

  • SVG vector system, with improved tools and workflow
  • New text tool
  • Python scripting
  • SVG import/export
  • Improved palette docker
  • Bigger brush sizes
  • Improved brush editor
  • Refactored saving and exporting: saving happens in the background, and export shows warnings when your file contains features that cannot be saved to a given file format
  • A fast colorize brush
  • The default pixel brush is much faster on systems with many cores
  • Lots of user interface polish

And there’s much more, but please download the builds, or build Krita and see for yourself!

One thing that is still in progress is updating the set of default brush presets: those aren’t in the beta yet, but they are in the nightly builds.

Platform Notes

Windows

Linux

  • The AppImage does not contain support for audio in animations
  • The AppImage does not have Python scripting
  • The AppImage does include the latest gmic-qt release

OSX

  • The app bundle does not contain gmic-qt
  • The app bundle does not contain Python scripting
  • The app bundle does not have support for importing PDF files

Download

Windows

Note for Windows users: if you encounter crashes, please follow these instructions to use the debug symbols so we can figure out where Krita crashes.

Linux

(If, for some reason, Firefox thinks it needs to load this as text: to download, right-click on the link.)

When it is updated, you can also use the Krita Lime PPA to install Krita 4.0 beta 1 on Ubuntu and derivatives.

OSX

Note: the gmic-qt and pdf plugins are not available on OSX.

Source code

md5sums

For all downloads:

Key

The Linux appimage and the source tarball are signed. You can retrieve the public key over https here:
0x58b9596c722ea3bd.asc
. The signatures are here.

Support Krita

Krita is a free and open source project. Please consider supporting the project with donations or by buying training videos or the artbook! With your support, we can keep the core team working on Krita full-time.

January 10, 2018

Phoning home after updating firmware?

Somebody made a proposal on the fwupd mailing list that the machine running fwupd should “phone home” to the LVFS with success or failure after the firmware update has been attempted.

This would let the hardware vendor that uploaded firmware know there are problems straight away, rather than waiting for thousands of frustrated users to file bugs. The report should needs to contain something that identifies the machine and a boolean, and in the event of an error, enough debug information to actually be useful. It would obviously involve sending the users IP address to the server too.

I ran a poll on my Google+ page, and this was the result:

So, a significant minority of people felt like it stepped over the line of privacy v.s. pragmatism. This told me I couldn’t just forge onward with automated collection, and this blog entry outlines what we’ve done for the 1.0.4 release. I hope this proposal is acceptable to even the most paranoid of users.

The fwupd daemon now stores the result of each attempted update in a local SQLite database. In the event there’s a firmware update that’s been attempted, we now ask the user if they would like to upload this information to the LVFS. Using GNOME this would just be a slider in the control center Privacy panel, and I’ll leave it to the distros to decide if this slider should be on or off by default. If you’re using the fwupdmgr tool this is what it shows:

$ fwupdmgr report-history
Target:                  https://the-lvfs-server/lvfs/firmware/report
Payload:                 {
                           "ReportVersion" : 1,
                           "MachineId" : "9c43dd393922b7edc16cb4d9a36ac01e66abc532db4a4c081f911f43faa89337",
                           "DistroId" : "fedora",
                           "DistroVersion" : "27",
                           "DistroVariant" : "workstation",
                           "Reports" : [
                             {
                               "DeviceId" : "da145204b296610b0239a4a365f7f96a9423d513",
                               "Checksum" : "d0d33e760ab6eeed6f11b9f9bd7e83820b29e970",
                               "UpdateState" : 2,
                               "Guid" : "77d843f7-682c-57e8-8e29-584f5b4f52a1",
                               "FwupdVersion" : "1.0.4",
                               "Plugin" : "unifying",
                               "Version" : "RQR12.05_B0028",
                               "VersionNew" : "RQR12.07_B0029",
                               "Flags" : 674,
                               "Created" : 1515507267,
                               "Modified" : 1515507956
                             }
                           ]
                         }
Proceed with upload? [Y|n]: 

Using this new information that the user volunteers, we can display a new line in the LVFS web-console:

Which expands out to the report below:

This means vendors using the LVFS know first of all how many downloads they have, and also the number of success and failures. This allows us to offer the same kind of staged deployment that Microsoft Update does, where you can limit the number of updated machines to 10,000/day or automatically pause the specific firmware deployment if > 1% of the reports come back with failures.

Some key points:

  • We don’t share the IP address with the vendor, in fact it’s not even saved in the MySQL database
  • The MachineId is a salted hash of your actual /etc/machine-id
  • The LVFS doesn’t store reports for firmware that it did not sign itself, i.e. locally built firmware archives will be ignored and not logged
  • You can disable the reporting functionality in all applications by editing /etc/fwupd/remotes.d/*.conf
  • We have an official GDPR document too — we’ll probably link to that from the Privacy panel in GNOME

Comments welcome.

January 09, 2018

Libre Graphics Meeting + SCaLE 2018

This year is starting off great with not one, but two libre graphics oriented events (one in Europe and the other in North America - for even better coverage)! Now you can double your opportunities to visit like-minded Free Software graphic artists, photographers, and developers.

Libre Graphics Meeting 2018 in Seville

LGM Logo

Come and join us for the 13th annual Libre Graphics Meeting (LGM) being held April 26–30 in Seville, Spain! LGM is a wonderful opportunity for artists, developers, and contributors to meet face-to-face and share/learn from each other. Several GIMP developers will be holding an annual team meeting there (as usual), so come and say hello!

The main programme this year is focusing on:

  • Technical presentations and workshops for developers.
  • Showcases of excellent work made using libre graphics tools.
  • New tools and workflows for graphics and code.
  • Reflections on the activities of existing Free/Libre and Open Source communities.
  • Reflections and practical sessions on promoting the philosophy and use of Libre Graphics tools.

Libre Graphics at Southern California Linux Expo

SCaLE 16x Logo

For the first time this year there will be a dedicated Libre Graphics track at the Southern California Linux Expo (SCaLE) 16x!

SCaLE 16x will be held March 8–11 in Pasadena, California (USA) at the Pasadena Convention Center, and the Libre Graphics track in particular will be on Friday, March 9th.

Pat David will be presenting on the state of GIMP (looking forward) and including GIMP as part of a photography workflow.

There is a Call for Participation out for those that would be interested in talking more about their art or involvement in the Free Software graphics world. If you’d like to participate submit a proposal to graphics-cfp@socallinuxexpo.org!

January 08, 2018

LibreOffice Vanilla 5.4.4 released on the Mac App Store

Collabora has now released LibreOffice Vanilla 5.4.4 on the Mac App Store. It is built from the official LibreOffice 5.4.4 sources. If you have purchased LibreOffice Vanilla earlier from the App Store, it will be upgraded in the normally automatic manner of apps purchased from the App Store.

LibreOffice Vanilla from the Mac App Store is recommended to Mac users who want LibreOffice with the minimum amount of manual hassle with installation and upgrades. If you don't mind that, by all means download and install the build from TDF instead.

We would have loved to continue to include a link to the TDF download site directly in the app's description, as we have promised, but we were not allowed to do that this time by Apple's reviewer.

Because of the restrictions on apps distributed in the App Store, features implemented in Java are not available in LibreOffice Vanilla. Those features are mainly the HSQLDB database engine in Base, and some wizards.

This time we include the localised help files, as there were some issues in accessing the on-line help.

Since the LibreOffice Vanilla 5.2 build that was made available in the Mac App Store in September 2016, there have been a few Mac-specific fixes, like the one related to landscape vs. portrait mode printing on Letter paper. There are more Mac-specific bugs in Bugzilla that will be investigated as resources permit.

Some fine-tuning to the code signing script has been necessary. For instance, one cannot include shell scripts in the Contents/MacOS subfolder of the application bundle when building for upload to the App Store. This is because the code signatures for such shell scripts would be stored as extended attributes and those won't survive the mechanism used to upload a build to the App Store for review and distribution. (For other non-binary files, in the Resources folder, signatures are stored in a separate file.)

We also have made sure the LibreOffice code builds with a current Xcode (and macOS SDK).

Interview with Emily K. Mell

Could you tell us something about yourself?

I’m a 30-year-old housewife and mother living in New Mexico in the United States. I’ve always had an interest in becoming a storyteller, and visual art is the most appealing way for me to do it.

Do you paint professionally, as a hobby artist, or both?

Right now I would call myself a hobbyist, as I’ve never published anything professionally. I hope for that to change. What I need is to somehow find the time and energy to crank out images at high volume!

What genre(s) do you work in?

I’ve always been interested primarily in fantasy, although much of my work has consisted of humorous cartoons. Humor creeps into whatever I try to make whether or not I intend for it to be there. My goal for the next decade is to begin a serial, consistent fantasy world as portrayed through cartoons.

Whose work inspires you most — who are your role models as an artist?

My first and largest influences are the old-fashioned Disney animated films from the 20th century, as exemplified by the work of artists like Bill Tytla. I also consider cartoonists and illustrators like Bill Watterson, Maurice Sendak, and Hal Foster to be very influential on my style. Fantasy illustrators like Frank Frazetta inspire me on an emotional level.

How and when did you get to try digital painting for the first time?

I began using digital tools purely as a form of practice a few years ago. To me it was very casual; I was figure drawing and making fan art referencing in-jokes from a favorite podcast of mine called “We Hate Movies.” I didn’t intend to pursue it seriously, but found myself returning to it more and more often.

What makes you choose digital over traditional painting?

Traditional painting definitely produces the most beautiful results, but it’s a pain in the neck. Not only does it take up more time and physical space, it’s far more expensive and wasteful. I don’t like the “digital look,” and it will take a lot of practice to minimize that. However, I believe that real artists should never blame their tools for any failure.

I’d almost always drawn pictures with only a pencil before, but digital painting allows me to ink and color my images much more easily. It also opens up options for experimentation when I can simply recolor anything I don’t like. The hardest part has been trying to learn the kind of control with a stylus that I have with my own hand.

How did you find out about Krita?

My husband is an engineer who works frequently with Linux and is familiar with the open-source world. He suggested Krita to me when I was looking for a digital painting program not called Photoshop.

What was your first impression?

Krita was the most professional-looking Photoshop alternative that I’d come across. It also played nicely with my stylus and tablet in a way that some other software didn’t. Krita did have some bugginess and crashing, though.

What do you love about Krita?

That it’s free! I think it’s remarkable that the open-source community could create something of this quality without a money spigot. Given Adobe’s outrageous pricing scheme for Photoshop, you’d think that software like this couldn’t exist anywhere else. Krita is a much better option.

What do you think needs improvement in Krita? Is there anything that really annoys you?

Bugs and crashes come with the territory in open-source projects, but those are likely to be reduced over time. The real problem is how inaccessible Krita is to lay people. When I was looking to download the program for the first time, I had to follow a completely unintuitive chain of links that, to somebody like me, appeared to be made up of Seussian nonsense words (AppImages for cats? What’s a Gentoo? Why is it bloody?). Idiots like myself just want a giant button that says “GET THE LATEST STABLE VERSION OF KRITA HERE!” The way things are now, the less technically literate will give up on trying Krita before they have even started.

What sets Krita apart from the other tools that you use?

Krita has a great community of support that will ensure that it gets better year by year. It has the right priorities, i.e. essential tools like layers and brushes get implemented before more obscure features. Other than occasional crashes, I can just jump in and use it.

If you had to pick one favourite of all your work done in Krita so far, what would it be, and why?

I made this page in Krita in order to see whether I could use it for a webcomic I plan on creating. The experiment was a success, and I believe that my skills will continue to grow. This proves that I can use Krita to tell the kinds of stories that I have plans for.

What techniques and brushes did you use in it?

I mostly use a basic round brush that I resize accordingly. Line brushes are convenient for separating the panels. I think that skills are more important than tools, and I want to train myself to use the simplest tools that I can.

Where can people see more of your work?

I’m shy about publishing, but I realize that this is something I need to change about myself. Once I’ve worked up a sizable volume of content, which should be within the next year, I will be posting a regular webcomic called The Unknown Engine.

Anything else you’d like to share?

I’m grateful that the Internet and the open-source movement affords artists like me the opportunity to create and publish in new ways. I often think about Bill Watterson’s long fight with his syndicates, and how different it would have been for him if he was able to publish Calvin and Hobbes on the Internet under his own control. I’m nowhere near his level of course, but I want to improve my own skills until I get there.

January 04, 2018

SMEP emulation in PTI

An nice additional benefit of the recent Kernel Page Table Isolation (CONFIG_PAGE_TABLE_ISOLATION) patches (to defend against CVE-2017-5754, the speculative execution “rogue data cache load” or “Meltdown” flaw) is that the userspace page tables visible while running in kernel mode lack the executable bit. As a result, systems without the SMEP CPU feature (before Ivy-Bridge) get it emulated for “free”.

Here’s a non-SMEP system with PTI disabled (booted with “pti=off“), running the EXEC_USERSPACE LKDTM test:

# grep smep /proc/cpuinfo
# dmesg -c | grep isolation
[    0.000000] Kernel/User page tables isolation: disabled on command line.
# cat <(echo EXEC_USERSPACE) > /sys/kernel/debug/provoke-crash/DIRECT
# dmesg
[   17.883754] lkdtm: Performing direct entry EXEC_USERSPACE
[   17.885149] lkdtm: attempting ok execution at ffffffff9f6293a0
[   17.886350] lkdtm: attempting bad execution at 00007f6a2f84d000

No crash! The kernel was happily executing userspace memory.

But with PTI enabled:

# grep smep /proc/cpuinfo
# dmesg -c | grep isolation
[    0.000000] Kernel/User page tables isolation: enabled
# cat <(echo EXEC_USERSPACE) > /sys/kernel/debug/provoke-crash/DIRECT
Killed
# dmesg
[   33.657695] lkdtm: Performing direct entry EXEC_USERSPACE
[   33.658800] lkdtm: attempting ok execution at ffffffff926293a0
[   33.660110] lkdtm: attempting bad execution at 00007f7c64546000
[   33.661301] BUG: unable to handle kernel paging request at 00007f7c64546000
[   33.662554] IP: 0x7f7c64546000
...

It should only take a little more work to leave the userspace page tables entirely unmapped while in kernel mode, and only map them in during copy_to_user()/copy_from_user() as ARM already does with ARM64_SW_TTBR0_PAN (or CONFIG_CPU_SW_DOMAIN_PAN on arm32).

© 2018, Kees Cook. This work is licensed under a Creative Commons Attribution-ShareAlike 3.0 License.
Creative Commons License

January 01, 2018

24 Free Goddess Gifs

Goddess_gif_small_2 Goddess_gif_small_1 Goddess_gif_small_3Goddess_gif_small_4 Goddess_gif_small_5 Goddess_gif_small_6Goddess_gif_small_11 Goddess_gif_small_7 Goddess_gif_small_8Goddess_gif_small_9 Goddess_gif_small_19 Goddess_gif_small_10Goddess_gif_small_15 Goddess_gif_small_14 Goddess_gif_small_20Goddess_gif_small_12 Goddess_gif_small_13 Goddess_gif_small_16Goddess_gif_small_17 Goddess_gif_small_18 Goddess_gif_small_22Goddess_gif_small_23 Goddess_gif_small_21 Goddess_gif_small_24

Here are 24 individual goddess gifs to use for whatever. Free Culture. No permission needed. Go crazy. I love you.

Share/Bookmark

flattr this!

Blender projects in 2018 to look forward to

The blender.org project is very lucky with attracting talent –great developers working together with fantastic artists. It’s how Blender manages to stand out as a successful and highly functional free & open software project. In this post I want to thank everyone for a wonderful Blender year and give a view at all of the exciting things that are going to happen– in 2018! (Fingers crossed :)

Eevee

In 2016 it was just an idea, having an interactive viewport in Blender with rendering quality at PBR levels. Last year this project took off in ways beyond expectation – everyone should have seen the demos by now.

Eevee in Blender

Early in 2018 animation support will come back (with support for modifiers), with as highlight OpenSubdiv support (GPU based adaptive subdivision surfaces).

Read about the Eevee roadmap here.

Grease Pencil

Blender is an original innovator in this area –providing a fully functional 2D animation tool in a 3D environment. You have to see it to believe it –it’s a mindblowing workflow for animators and story artists.

Grease Pencil

In Q1 of 2018 the short film “Hero” will be finished as proof-of-concept for the new workflow and tools of Grease Pencil in 2.8x.

You can read the latest status report here.

Workflow & “Blender 101”

Optimizing and organizing one’s working environment can significantly improve the workflow in 3D applications. We can’t make everyone happy with a single Blender configuration anymore. This is where the new Workspaces and Application Templates come in. In Q1 and Q2 of 2018 the first prototypes for radically configured simple Blenders are going to be made (a.k.a. the Blender 101 project).

Meanwhile work continues on usability and configurations in a daily production environment. Blender’s latest Open Movie “Spring” is going to be used for this.

Blender 2.8x is also getting a complete new layer system, allowing to organize your scenes in advanced ways. A Scene can have unlimited amount of layers (= drawings or renders),  unlimited amounts of collections and per collection render settings and overrides.

Visit the code.blender.org blog to read more about it.

New UI theme

No, there are no pictures yet! But one of the cool things of releasing a massive update is to also to update the looks. Nothing radical, just to make it look fresh and to match contemporary desktop environments. We’re still using the (great) design from 2009-2010. In computer years, that’s a century ago! Work on this should start Q1 and get finalized before Q2 ends. Contributions welcome (check ‘get involved’).

Cycles

In 2017 we saw the rise of AMD GPUs. Thanks to a full time developer who worked on OpenCL for a year, Blender is now a good choice for use on AMD hardware. For 2018 we want to work on solving the kernel compiling waiting time.

The Daily Dweebs

Cycles is now one of the most popular areas for developers to work in. Most of these are doing this as part of their daytime job – to make sure Cycles stays an excellent choice for production rendering. Expect in 2018 a lot of high quality additions and especially ways to manage fast renders.

Read more about the Cycles Roadmap here.

Blender Game Engine

One of Blender’s best features is that it’s a complete integrated 3D creation suite –enabling artists to create projects from concept to final edits or playback. Unfortunately the game engine has fallen behind in development– not getting the focus or development time it needs. There are many reasons for it, but one of these is that the code base for BGE is too much separated from the rest of Blender. That means that newly added Blender features need to be ported over to the engine to work.

Blender Game Engine

For the 2.8 project we want to achieve a better integration of BGE and Blender itself. The Eevee project has proven already how important real-time tools are and how well this can work for interactive 3D design and game creators.

That being said, outside of blender.org interesting Blender-related development for game engines happens too. Check out the Blender fork UPBGE for example, or the fascinating Armory Engine (see image above, it’s written in Haxe and Kha). And don’t forget the open source WebGL environments Blend4Web and Verge3D.

Assets and presets

Another ‘2.8 workflow’ related feature: we are working on better managing your files and 3d assets. Partially it’s for complex production setup, partially it’s also about configuring your Workspaces with nice visible presets – lists of pictures of shaders or primitives for example, ready to be dragged & dropped or selected.

Asset engine preview

More information can be found here, the planning for asset management and overrides.

Viewport Compositing

An important design aspect of Blender’s new viewport system is that each engine is drawing in its own buffer. These buffers then get composited in real-time.

Blender 267 splash screen

To illustrate how fast it is: in 2.8x the “Overlay engine” is using real-time compositing in the viewport (to draw selections or widgets).

When 2.8x is ready to get out of beta, we will also check on how to allow (node based) real time compositing in viewports. That then is the final step to fully have replaced the old “Blender Internal” render engine with an OpenGL based system.

This will especially be interesting for the Non-Photo-Realistic rendering enthusiasts out there. Note – FreeStyle rendering will have to fully recoded for 2.8. That’s still an open issue.

Modifiers & Physics upgrade

Blender’s modifier code is getting fully rewritten for 2.8. That’s needed for the new dependency graph system (threadable animation updates and duplication of data).

Blender Physics

A nice side effect of this recode is that all modifiers then will be ‘node ready’. We expect first experiments with modifier nodes to happen in 2018. Don’t get too excited yet, it’s especially the complexity of upgrading the old particle and hair system that’s making it a very hard project to handle.

An important related issue here is how to handle “caches” well (i.e. generated mesh data by modifiers or physics systems). This needs to be saved and managed properly – which is what the dependency graph has to do as well. As soon that’s solved we can finally merge in the highly anticipated Fracture Modifier branch.

Animation tools

Blender’s armature and rigging system is based on a design from the 90ies. It’s a solid concept, but it’s about time to refresh and upgrade it. When Blender 2.8x gets closer to beta I want to move my focus on getting a project organized (and funded) to establish a small team of developers on animation tools for the next decade – Animation 2020! Contact me if you want to contribute.

Discourse forums

Improving onboarding for new developers is on our wish list already for years. There are several areas we should do better – more swiftly handle reviews for provided patches and branches for example.

Discourse Forum

We also often hear that Blender developer channels are hard to find or not very accessible. The blender.org teams  still mainly use IRC chat and MailMan mailing lists for communciation.

In January we will test a dedicated blender.org developer forum using Discourse (fully free/open software). This forum will focus on people working with Blender’s code, developer tools and anything related to becoming a contributor. If this experiment works well we can expand it to a more general “get involved” website (for docs, educators, scientists, conferences, events).
However, user questions, feature requests – would be off topic, there are better places that handle this.

20th anniversary of first public Blender release

Oh yes! Today is exactly 20 years ago that I released the first Blender version in public – only for the Silicon Graphics IRIX platform.

Blender on IRIX

A FreeBSD and Linux version were made a couple of months after.

All as “freeware” then, not open source. I first had to learn the lesson of bursting internet bubbles before going fully open!

Blender 2.80 beta release

Originally planned for Q2 this year… luckily that quarter lasts until July 1st. All depends on how well the current projects go the coming months. But if it’s not July first, then at least we have…

SIGGRAPH, Vancouver

The largest annual 3D CG event is at August 12-16 this year. We aim at a great presence there and certainly it’s a great milestone to showcase 2.80 there!

Open issues

The 2.8 team tries to keep focus – not to do too many things at once and to finish what’s being worked on in the highest usable quality possible. That means that some topics are being done first, and some later. The priorities for 2.8 have been written down in this mail to the main developers list.

We can still use a lot of help. Please don’t hesitate to reach out – especially when workflow and usability are your strength! But we can use contributors in many ‘orphaned’ areas: such as Booleans, Video editor, Freestyle render, Particles, Physics caching, Hair, Nurbs… but also to work on better integration with Windows and MacOS desktop environments.

Credits

An important part of the blender.org project are the studios and companies who contribute to Blender.

Special thanks goes to Blender Foundation (development fund grants), Blender Institute/Animation Studio (hiring 3-5 devs), Tangent Animation (viewport development), Aleph Objects (from Lulzbot printers, supporting Blender 101), Nimble Collective (Alembic), AMD (Cycles OpenCL support), Intel (seeding hardware, Cycles development), Nvidia (seeding hardware), Theory Animation and Barnstorm VFX (Cycles development, VFX pipeline).

Special thanks also to the biggest supporters of the Development Fund: Valve Steam Workshop and Blender Market.

Ton Roosendaal, Chairman Blender Foundation

December 30, 2017

GIMP and GEGL in 2017

When you say you mostly do bugfixing now, seven kinds of new features will crawl under your bed and bite your silly toes off. If we were to come up with a short summary for 2017, it would be along those very lines.

So yes, we ended up with more new features that, however, make GIMP faster and improve workflows. Here’s just a quick list of the top v2.10 parole violators: multi-threading via GEGL, linear color space workflow, better support for CIE LCH and CIE LAB color spaces, much faster on-canvas Warp Transform tool, complete on-canvas gradients editing, better PSD support, metadata viewing and editing, under- and overexposure warning on the canvas.

All of the above features (and many more) are available in GIMP 2.9.8 released earlier this month. We are now in the strings freeze mode which means there will be very few changes to the user interface so that translators could safely do their job in time for the v2.10 release.

Everyone is pretty tired of not having GIMP 2.10 out by now, so we only work on bugs that block the v2.10 release. There are currently 25 such bugs. Some are relatively easy to fix, some require more time and effort. Some have patches or there is work in progress, and some need further investigation. We will get there faster, if more people join to hack on GIMP.

Speaking of which, one thing that has changed in the GIMP project for the better this year is the workload among top contributors. Michael Natterer is still responsible for 33% of all GIMP commits in the past 12 months, but that’s a ca. 30% decrease from the last year. Jehan Pagès and Ell now have a 38% share of all contributions, and Øyvind Kolås tops that with his 5% thanks to the work on layers blending/compositing and linear color space workflow in GIMP.

In particular, Ell committed the most between 2.9.6 and 2.9.8, implemented on-canvas gradients editing, introduced other enhancements, and did a lot of work on tuning performance in both GIMP and GEGL. We want to thank him especially for being the most prolific developer of GIMP for this last development release!

Another increasingly active contributor in the GEGL project is Debarshi Ray who uses the library for his project, GNOME Photos. Debarshi focused mostly on GEGL operations useful for digital photography such as exposure and shadows-highlights, and did quite a lot of bugfixing. We also got a fair share of contributions from Thomas Manni who added some interesting experimental filters like SLIC (Simple Linear Iterative Clustering) and improved existing filters.

Changes in GEGL and babl in 2017 included (but are not limited to) 15 new filters, improvements in mipmap processing and multi-threading computations, a video editing engine called gcut, more fast paths to convert pixel data between various color spaces, support for custom RGB primaries and TRC, ICC color profiles parsing and generation, and numerous bugfixes.

At least some of the work done by Øyvind Kolås on both GEGL, babl, and GIMP this year was sponsored by you, the GIMP community, via Patreon and Liberapay platforms. Please see his post on 2017 crowdfunding results for details and consider supporting him. Improving GEGL is crucial for GIMP to become a state-of-the art professional image editing program. Over the course of 2017, programming activity in GEGL and babl increased by 120% and 102% respectively in terms of commits, and we’d love to see the dynamics keep up in 2018 and onwards.

Even though the focus of another crowdfunded effort by Jehan Pagès and Aryeom Han is to create an animated short movie, Jehan Pagès contributed roughly 1/5 of code changes this year, fixing bugs, improving painting-related features, maintaining GIMP official Flatpak, and these statistics don’t even count the work on a much more sophisticated animation plug-in currently available in a dedicated Git branch. Hence supporting this project results in better user experience for GIMP users. You can help fund Jehan and Aryeom on Liberapay, Patreon or Tipeee. You can also read their end-of-2017 report.

We also want to thank Julien Hardelin who has been a great help in updating the user manual for upcoming v2.10, as well as all the translators and people who contributed patches. Moreover, we thank Pat David for further work on the new website, and Michael Schumacher for tireless bug triaging. They all don’t get nearly as much praise as they deserve.

Happy 2018!

December 29, 2017

26 Goddesses

I’ve been painstakingly hand-removing backgrounds from photos in GIMP, so I can use them in Moho.Goddesses_ancient2_4

Goddesses_ancient2_9

Queen of the Night

 

Snake_Dancer_12

Here’s my collection:

Goddesses_ancient - Frame 0.2

Goddesses_ancient - Frame 0.3

Goddesses_ancient - Frame 0.4

Goddesses_ancient2 - Frame 0

Share/Bookmark

flattr this!

Looking back, looking forward

First and foremost, 2017 ends well. We will end this year putting Krita 4.0 in string freeze, which means a release early next year! In 2017, we’ve released several versions of Krita 3.x. We’ve gained a lot of new contributors with great contributions to Krita. We’ve got money in the bank, too. Less than last year, but sales on the Windows Store help quite a bit! And development fund subscriptions have been steadily climbing, and we’re at 70 subscribers now! We’ve also done a great project with Intel, which not only brought some more money in, but also great performance improvements for painting and rendering animations.

It’s been a tough year, though! Our maintainer had only just recovered from being burned out from working full-time on Krita and on a day job when the tax office called… The result was half a year of stress and negotiations, ending in a huge tax bill and a huge accountant’s bill. And enough uncertainty that we couldn’t have our yearly fund raiser, and enough extra non-coding work that the work on the features funded in 2016 took much, much more time than planned. In the period when we were talking to the tax office, until we could go public, Boudewijn and Dmitry were supported by members from the community; without that support the project might not have survived.

But then, when we could go public with our problems, the response was phenomenal. At that point, we were confident we would survive anyway, with the work we were doing for Intel, the Windows Store income and private savings, but it would have been extremely tight.  The community rallied around us magnificently, and then Private Internet Access (who also sponsor KDE, Gnome, Blender and Inkscape, among others) contacted us with their decision to pay the bill!

From nearly broke, we went to be in a position to start planning again!

We reported about those plans before, but to recap:

  • We will to release Krita 4.0 with Python scripting, SVG vector layers, a new text tool, the stacked brushes feature (now renamed to masking brush) and the lazy coloring brush, and many more features. String freeze December 31st 23:59:59, release planned for March!
  • We want to spend next year working on bug fixes, performance improvements and polish
  • But there will also be work on a new reference images tool, improved session management and other things.
  • We will look into the possibility of porting Krita to Android and iOS, though the first attempts have not been promising.
  • We will do another fund raiser, though whether that will be a kickstarter hasn’t been decided yet.
  • After being constrained from attending open source conferences in 2016 and 2017, we intend to have at least someone present at the Libre Graphics Meeting and Akademy. We shouldn’t get disconnected from our roots!

Akademy is the yearly KDE community conference, and Krita has always been part of the KDE community. And KDE has always been more than a desktop environment for Linux and other Unix-like operating systems. As a community, KDE offers an environment where projects like Krita can flourish. Every developer in the KDE community can work on any of the KDE projects; the level of trust in each other is very high.

These days, judging by the number of bugs reported and closed, Krita is the second-most used KDE project, after the Plasma desktop shell. Without KDE, Krita wouldn’t be where it’s now. Without KDE, and the awesome job it’s volunteer sysadmins are doing, we wouldn’t have working forums, continuous integration, bug trackers or a project management platform. Sprints would be far more difficult to organize, and, of course, Krita depends heavily on a number of KDE framework libraries that make our coding life much easier. KDE is currently having the annual End of Year Fundraiser!

Our contributor sprint in 2016 was partly sponsored by KDE as well. With all the bother, it looked like we wouldn’t meet up in 2017. But with the project back on a sound footing, we managed to have a small sprint in November after all, and much vigorous discusion was had by all participants, ending up with firm plans for the last few 4.0 features that we were working on. Next year, we intend to have another big contributor sprint as well.

And, of course, lots of lovely releases, bug fixes, features, artist interviews, documentation updates, and the please of seeing so many people create great art!

December 28, 2017

First Krita painting course in Colombia

Krita is not widely known in Latin America. In Colombia, we found that people are interested in knowing more about how to use it. This year, in April 2017, the program of the Latin American Free Software Install Fest included a workshop by David Bravo about Krita. The workshop was fully booked and inspired us to create this course.

colombia krita course participants

Left to right: Mateo Leal, Angie Alzate, David Bravo (teacher), Lina Porras, Lucas Gelves, Juan Pablo Sainea, Javier Gaitán

During 4 sessions of 3 hours each, David Bravo guided a group of six students through their first steps in Krita, including sketch, canvas, digitalization, lines, curves and brush, light and shadow, digital color, painting and color palette, texture, effects, exporting files for digital media and printing.

David Bravo and his drawing

David Bravo (front). The projected drawing is his work.

This course was made possible by the cooperation of three organizations: Onoma Project, Corre Libre Foundation and Ubuntu Colombia. The cost for the students was about 16 USD; all of the proceeds were donated to the Krita Foundation.

Lucas Gelves' work

Lucas Gelves teaching himself to draw.

We think that we can offer an intermediate course in 2018. And of course we want to say thank you to the Krita Foundation for sending gifts for the course students and for staying in touch with us. We hope to cooperate in the near future for future courses!

onoma logoDavid Bravo is a digital and multimedia designer from Colegio Mayor de Cundinamarca, currently working in multimedia freelance projects with a focus on traditional animation, 3D and visualization in virtual environments. He is also the leader of the Onoma Project, an online free platform that is under development. The main objective of this project is to provide tools for easy and secure learning of FLOSS for design.

Ubuntu Colombia logoUbuntu Colombia acts as coordinator and communicator of the course. Ubuntu Colombia is a community with 12 years of history in spreading Ubuntu and FLOSS in Colombia; the Krita course was part of this year’s efforts of the community to promote education on FLOSS tools, as were LaTeX courses and LPIC preparation courses.

Corre Libre Foundation is an NGO created in 2008. Its objectives are:
– to promote the creation of free/open knowledge
– to sponsor free technological projects with social impact
– to promote and spread the use and development of technologies that contribute to human freedom
– to promote and spread collaborative work.

They support Orfeo, which is documentary software. For this course they provided a place to work, which would otherwise have been too difficult and expensive to find in our city.

December 25, 2017

Interview with Rositsa Zaharieva

Could you tell us something about yourself?

My name is Rositsa (also known as Roz) and I’m somewhat of a late blooming artist. When I was a kid I was constantly drawing and even wanted to become an artist. Later on I chose a slightly different path for my education and career and as a result I now have decent experience as a web and graphic designer, front end developer and copywriter. I am now completely sure that I want to devote myself entirely to art and that’s what I’m working towards.

Do you paint professionally, as a hobby artist, or both?

I mainly work on personal projects. I have done some freelance paintings in the past, though. I’d love to paint professionally full time sometime soon, hopefully for a game or a fantasy book of some sort.

What genre(s) do you work in?

I prefer fantasy most of all and anything that’s not entirely realistic. It has to have some magic in it, something from another world. That’s when I feel most inspired.

Whose work inspires you most — who are your role models as an artist?

I’m a huge fan of Bill Tiller’s work for The Curse of Monkey Island, A Vampyre Story and Duke Grabowski, Mighty Swashbuckler! Other than him I’m following countless other artists on social networks and use their work to get inspired. Also, as a member of a bunch of art groups I see great
artworks from artists I’ve never heard of every single day and that’s also a huge inspiration.

How and when did you get to try digital painting for the first time?

My first encounter with digital painting was in 2006-2007 on deviantART but it wasn’t until 2010-2011 when I finally got my precious Wacom Bamboo tablet (which I still use by the way!) that I could finally begin my own digital art journey for real.

What makes you choose digital over traditional painting?

Digital painting combines my two loves – computers and art. It only seems logical to me that I chose it over traditional art but back then I didn’t give it that much thought – I just thought how awesome all the paintings I was seeing at the time were and how I’d love to do that kind of art myself. I’ve since come to realize that one doesn’t really have to choose one or the other – I find doing traditional art every once in a while incredibly soothing, even though I’ve chosen to focus on digital art as my career path.

How did you find out about Krita?

I think I first got to know about Krita from David Revoy on Twitter some years ago, but it wasn’t until this year when I finally decided to give it a try.

What was your first impression?

My first impression was just WOW. I thought “OMG, it’s SO similar to Photoshop but has all these features in addition and it’s FREE!” I was really impressed that I could do all that I was used to do in Photoshop but in a native Linux application and free of charge.

What do you love about Krita?

Exactly what I mentioned above. I’m still kind of a newbie with Krita so there’s not so much to tell but I’m sure I’m yet to discover a lot more to love as time goes by.

What do you think needs improvement in Krita? Is there anything that really annoys you?

I’d like to see an improved way of working with the Bezier Curve Selection Tool as I use it a lot but am having trouble doing perfect selections at one go. I’d really like to be able to variate between corner anchor points and curves on the fly, as I’m creating the selection, instead of creating a somewhat messy selection and then having to go back and clean it up by adding and subtracting parts of it until it looks the way I’d intended. That would certainly save me a lot of time.

What sets Krita apart from the other tools that you use?

That it’s free to use but not any less usable than the most popular paid applications of the sort! Also, the feeling I get whenever I’m involved with Krita in any way – be it by reading news about it, interacting on social media or painting with it. I’m just so excited that it exists and grows and is getting better and better. I feel somewhat proud that I’m contributing even in the tiniest way.

If you had to pick one favourite of all your work done in Krita so far, what would it be, and why?

I love everything I’ve created in Krita so far. I don’t think it’s that much about the software you create a certain artwork with but rather allthe love you put in it as you’re creating it.

What techniques and brushes did you use in it?

I’m trying to use less brushstrokes and more colorful shapes as I paint. I mainly use the Bezier Curve Selection Tool, the Gradient Tool and a small set of Krita’s predefined brushes for my artworks. I have tried creating my own custom brushes but with little luck so far (I think I have much more reading and experimenting to do before I succeed).

Where can people see more of your work?

I have a portfolio website (in Bulgarian): www.artofroz.com; but you can find me on Facebook, Twitter, Behance, Artstation and a bunch of other places either as ArtofRoz, Rositsa Zaharieva or some combo/derivative of both.

Anything else you’d like to share?

I’d like to tell everyone that’s been using other software for their digital paintings to definitely give Krita a try, too. Not that other software is bad in any way, but Krita is awesome!

Saving a transparent PNG image from Cairo, in Python

Dave and I will be giving a planetarium talk in February on the analemma and related matters.

Our planetarium, which runs a fiddly and rather limited program called Nightshade, has no way of showing the analemma. Or at least, after trying for nearly a week once, I couldn't find a way. But it can show images, and since I once wrote a Python program to plot the analemma, I figured I could use my program to generate the analemmas I wanted to show and then project them as images onto the planetarium dome.

[analemma simulation] But naturally, I wanted to project just the analemma and associated labels; I didn't want the blue background to cover up the stars the planetarium shows. So I couldn't just use a simple screenshot; I needed a way to get my GTK app to create a transparent image such as a PNG.

That turns out to be hard. GTK can't do it (either GTK2 or GTK3), and people wanting to do anything with transparency are nudged toward the Cairo library. As a first step, I updated my analemma program to use Cairo and GTK3 via gi.repository. Then I dove into Cairo.

I found one C solution for converting an existing Cairo surface to a PNG, but I didn't have much luck with it. But I did find a Python program that draws to a PNG without bothering to create a GUI. I could use that.

The important part of that program is where it creates a new Cairo "surface", and then creates a "context" for that surface:

surface = cairo.ImageSurface(cairo.FORMAT_ARGB32, *imagesize)

cr = cairo.Context(surface)

A Cairo surface is like a canvas to draw on, and it knows how to save itself to a PNG image. A context is the equivalent of a GC in X11 programming: it knows about the current color, font and so forth. So the trick is to create a new surface, create a context, then draw everything all over again with the new context and surface.

A Cairo widget will already have a function to draw everything (in my case, the analemma and all its labels), with this signature:

    def draw(self, widget, ctx):

It already allows passing the context in, so passing in a different context is no problem. I added an argument specifying the background color and transparency, so I could use a blue background in the user interface but a transparent background for the PNG image:

    def draw(self, widget, ctx, background=None):

I also had a minor hitch: in draw(), I was saving the context as self.ctx rather than passing it around to every draw routine. That means calling it with the saved image's context would overwrite the one used for the GUI window. So I save it first.

Here's the final image saving code:

   def save_image(self, outfile):
        dst_surface = cairo.ImageSurface(cairo.FORMAT_ARGB32,
                                         self.width, self.height)

        dst_ctx = cairo.Context(dst_surface)

        # draw() will overwrite self.ctx, so save it first:
        save_ctx = self.ctx

        # Draw everything again to the new context,
        # with a transparent instead of an opaque background:
        self.draw(None, dst_ctx, (0, 0, 1, 0))  # transparent blue

        # Restore the GUI context:
        self.ctx = save_ctx

        dst_surface.write_to_png("example.png")
        print("Saved to", outfile)

December 24, 2017

darktable 2.4.0 released

we’re proud to finally announce the new feature release of darktable, 2.4.0!

the github release is here: https://github.com/darktable-org/darktable/releases/tag/release-2.4.0.

as always, please don’t use the autogenerated tarball provided by github, but only our tar.xz. the checksums are:

$ sha256sum darktable-2.4.0.tar.xz
9d37388aee79d5ada71062bbac3cda612a61d1a781f6320b784b27308f3a1878 darktable-2.4.0.tar.xz
$ sha256sum darktable-2.4.0.dmg
70dcbec46c54f2006f2887b7ec1c9d748f9a726389d3b75cd5e081695e26394e darktable-2.4.0.dmg
$ sha256sum darktable-2.4.0-win64.exe
5b7b00a0bed8ea0d5ac45b0a0668f1998ad396e4bc3b5791e7a17f7c70b90f7c darktable-2.4.0.exe

when updating from the currently stable 2.2.x series, please bear in mind that your edits will be preserved during this process, but it will not be possible to downgrade from 2.4 to 2.2.x any more.

Important note: to make sure that darktable can keep on supporting the raw file format for your camera, please read this post on how/what raw samples you can contribute to ensure that we have the full raw sample set for your camera under CC0 license!

  • The maintainership of the RawSpeed library was transferred to the darktable project. The work on code cleanup, hardening, modernization, simplification and testing is ongoing.
  • Almost 3 thousand commits to darktable+rawspeed since 2.2.0
  • 273 pull requests handled
  • 340+ issues closed
  • Updated user manual is coming soon™

Gource visualization of git log from 2.2.0 to right before 2.4.0:

Hell Froze Over

  • As you might have read on our news post we finally ported darktable to Windows and intend to support it in the future. At the moment it’s still lacking a few features (for example there is no printing support), has a few limitations (tethering requires special drivers to be installed) and comes with its own set of bugs (TIFF import and export doesn’t support non-ASCII characters in file names). But overall we are confident that it’s quite usable already and hope you will enjoy it. A very special thanks goes to Peter Budai who finally convinced us to agree to the port and who did most of the work.

The Big Ones

  • A new module for haze removal
  • The local contrast module can now be pushed much further, it also got a new local laplacian mode
  • Add undo support for masks and more intelligent grouping of undo steps
  • Blending now allows to display individual channels using false colors
  • darktable now supports loading Fujifilm compressed RAFs
  • darktable now supports loading floating point HDR DNGs as written by HDRMERGE
  • We also added channel specific blend modes for Lab and RGB color spaces
  • The base curve module allows for more control of the exposure fusion feature using the newly added bias slider
  • The tonecurve module now supports auto colour adjustment in RGB
  • Add absolute color input as an option to the color look up table module
  • A new X-Trans demosaicing algorithm, Frequency Domain Chroma, was implemented.
  • You can now choose from pre-defined scheduling profiles for OpenCL
  • Speaking of OpenCL, darktable now allows to force-use OpenCL for a specific pixelpipe
  • XMP sidecar files are no longer written to disk when the content didn’t actually change. That mostly helps with network storage and backup systems that use files’ time stamps

New Features And Changes

  • Show a dialog window that tells when locking the database/library failed
  • Don’t shade the whole region on the map when searching for a location. Instead just draw a border around it.
  • Also in map mode: Clear the search list and map indicators when resetting the search module.
  • With OsmGPSMap newer than version 1.1.0 (i.e., anything released after that OsmGPSMap version) the map will show copyright info.
  • Running jobs with a progressbar (mostly import and export) will show that progress bar ontop the window entry in your task bar – if the system supports it. It should work on GNOME, KDE and Windows at least.
  • Add bash like string replacement for variables (export, watermark, session settings)
  • Add a preferences option to ask before removing empty dirs
  • The “colorbalance” module got a lot faster, thanks to SSE optimized code
  • Make gradient sliders a little more colorful
  • Make PNG compression level used for exporting configurable
  • On OSX, load single images from command line or via drag&drop in darkroom mode
  • Add an option to omit the intermediate tag hierarchy in exported files and only add the last level
  • In the watermark module, sort the list of SVG files and omit the file extension
  • Support XYZ as a proofing profile
  • Local contrast now got a new slider to set the midtone range
  • darktable got two new helper scripts (those are not installed by default, grab them from the sources)
    • One to purge thumbnails that no longer have an associated image in the database,
    • and a second script that uses inotify to watch a folder for new files to open them in a running darktable instance.
  • In the curve editors of base curve and tone curve you can now delete nodes with a right click and see coordinates of nodes while editing. Note that you can use keyboard modifiers ctrl and shift to change the precision of your changes
  • Creating a new instance of a module can now be done with a quick click of the middle mouse button on the multi-instance icon
  • New darktable installations on computers with more than 8 Gb of memory will now by default use half of that per module
  • Several background colors and the brush color are now configurable in the CSS
  • Some new cameras can bump the ISO level to insane highs. We try to follow as good as we can by no longer limiting it to 51200 in the GUI
  • Base curve and the highlights module now support multiple instances and use blending and masks
  • Having the 1 key toggle between 1 and 0 stars wasn’t very popular with many people. You can disable that extra feature and have it behave like the other rating shortcuts now
  • You can decide if you want to be asked before resetting the history stacks of images from the lighttable
  • The grain module was slightly changed to have a more pleasing, photographic-paper like appearance
  • Using the color look up table module you can now convert your images to monochrome, honoring the Helmholtz-Kohlrausch effect
  • Support basic import of Lightroom 7 settings
  • Change the styling of insensitive bauhaus widgets
  • Don’t hide the mode combobox in the exposure module, just disable it
  • Read primaries and whitepoint from .hdr files and default to those as the input color profile
  • Some more small improvements were made

Bugfixes

  • Fix the problem with rating images by accident when moving the mouse while typing an image size in the export module
  • Fix several oddities in folder and tag mode of the collect module
  • Print mode’s color profile settings no longer interact with the export module
  • Update the style lists when importing a style
  • Fix some bugs with multiple module instances used in a style
  • On OSX only the main window should be fullscreen, not the popups
  • Some speedups with VERY big libraries or having A LOT OF tags
  • Significantly speed up tagging many images
  • Fix searching locations using OpenStreetMap
  • Fix partial copies of large files in “import from camera”
  • Fix a crash in the import dialog when using Lua to add widgets there
  • Fix some false-positive warnings about another running darktable instance and it having locked the databases
  • No longer switch to the favourite modules group when duplicating one of its modules
  • Fix loading of XYZ files
  • Fix Lab export when the profile was set from the lighttable
  • Create temporary snapshot files with mode 0600 to stop other people looking at them
  • Fix several bugs with Wayland. However, there are still issues, so darktable will prefer XWayland
  • Google deprecated the Picasa Web API so it’s no longer possible to create G+ albums
  • Fix the default for sliders with target not being “red” in the channel mixer
  • Fix the removal of directories
  • Make the escape key cancel history dialogs
  • Block keyboard accels when editing camera controls
  • Properly delete XMP sidecars
  • Make sure that the rating set in darktable is used for the exported file, not something set inside the raw file
  • Don’t re-write all XMP files when detaching a tag
  • Sync XMPs when a tag is removed from the database
  • Sync XMPs after a tag is attached/detached via the Lua API
  • Bail out of darktable-cli when the XMP file is not readable
  • Show ratings on zoomable lighttable without a delay
  • Rely on CUPS color management when printing without configuring any color profile in darktable
  • Fix spurious segfault in local contrast
  • Make calls to exiv2’s readMetadata thread safe to not crash randomly
  • Properly read Lightroom XMPs on systems with , as the decimal separator
  • Fix setting the PNG bit depth from the gui
  • Many more bugs got fixed

Lua

  • darktable now uses Lua 5.3. The bundled copy got updated accordingly
  • Add dt.print_log. It’s like print_error but without the ERROR prefix
  • Reorder callback parameters for intermediate export image: add the actual image to the parameters of the event
  • Call lua post-import-image event synchronously
  • Add darktable.configuration.running_os to detect the OS darktable is running on
  • New widget type: section_label, adds a label which looks like a section change

Changed Dependencies

  • CMake 3.1 is now required
  • In order to compile darktable you now need at least gcc-5.0+/clang-3.4+
  • ZLIB is now required for the DNG Deflate compressed raw support
  • darktable now uses Lua 5.3

Camera support, compared to 2.2.0

Warning: support for Nikon NEF ‘lossy after split’ raws was unintentionally broken due to the lack of such samples. Please see this post for more details. If you have affected raws, please contribute samples!

Base Support

  • Canon EOS 200D
  • Canon EOS Kiss X9
  • Canon EOS Rebel SL2
  • Canon EOS 6D Mark II (sRaw1, sRaw2)
  • Canon EOS 77D
  • Canon EOS 9000D
  • Canon EOS 800D
  • Canon EOS Kiss X9i
  • Canon EOS Rebel T7i
  • Canon EOS M100
  • Canon EOS M5
  • Canon EOS M6
  • Canon PowerShot G9 X Mark II
  • Canon PowerShot SX40 HS (dng)
  • Fujifilm GFX 50S (compressed)
  • Fujifilm X-A3
  • Fujifilm X-E2S
  • Fujifilm X-E3 (compressed)
  • Fujifilm X-Pro2 (compressed)
  • Fujifilm X-T2 (compressed)
  • Fujifilm X-T20 (compressed)
  • Fujifilm X100F (compressed)
  • GITUP GIT2P (chdk-a, chdk-b)
  • Kodak EasyShare Z980
  • LG D855 (dng)
  • LG H815 (dng)
  • LG Nexus 5X (dng)
  • LG US996 (dng)
  • LG VS995 (dng)
  • Leica D-LUX (Typ 109) (4:3, 3:2, 16:9, 1:1)
  • Leica X2 (dng)
  • Nikon COOLPIX B700 (12bit-uncompressed)
  • Nikon D500 (14bit-uncompressed, 12bit-uncompressed)
  • Nikon D5600 (12bit-compressed, 12bit-uncompressed, 14bit-compressed, 14bit-uncompressed)
  • Nikon D7500 (12bit-compressed, 14bit-compressed)
  • Nikon D850 (12bit-compressed, 12bit-uncompressed, 14bit-compressed, 14bit-uncompressed)
  • Nikon LS-5000 (dng)
  • Nokia Lumia 1020 (dng)
  • Olympus E-M10 Mark III
  • Olympus E-M1MarkII
  • Olympus TG-5
  • Panasonic DC-FZ82 (4:3)
  • Panasonic DMC-FZ80 (4:3)
  • Panasonic DMC-FZ85 (4:3)
  • Panasonic DC-GH5 (4:3)
  • Panasonic DC-FZ91 (4:3)
  • Panasonic DC-FZ92 (4:3)
  • Panasonic DC-FZ93 (4:3)
  • Panasonic DC-TZ90 (4:3)
  • Panasonic DC-ZS70 (4:3)
  • Panasonic DMC-FZ330 (4:3)
  • Panasonic DMC-GF6 (16:9, 3:2, 1:1)
  • Panasonic DMC-TZ61 (4:3, 3:2, 1:1, 16:9)
  • Panasonic DMC-ZS40 (4:3, 3:2, 1:1, 16:9)
  • Panasonic DMC-TZ80 (4:3)
  • Panasonic DMC-TZ81 (4:3)
  • Panasonic DMC-TZ85 (4:3)
  • Panasonic DMC-ZS60 (4:3)
  • Pentax K-5 (dng)
  • Pentax K-r (dng)
  • Pentax K10D (dng)
  • Phase One IQ140
  • Samsung G920F
  • Samsung G935F
  • Samsung GX10
  • Sony ILCE-6500
  • Sony ILCE-7RM3
  • Sony ILCE-9

White Balance Presets

  • Canon EOS 6D Mark II
  • Fujifilm X-T20
  • Fujifilm X100F
  • Nikon 1 AW1
  • Nikon Coolpix A
  • Panasonic DMC-GX80
  • Panasonic DMC-GX85
  • Panasonic DMC-TZ100
  • Panasonic DMC-TZ101
  • Panasonic DMC-TZ110
  • Panasonic DMC-ZS110
  • Pentax K-3 II

Noise Profiles

  • Canon EOS 1300D
  • Canon EOS Kiss X80
  • Canon EOS Rebel T6
  • Canon EOS 5D Mark IV
  • Canon EOS 6D Mark II
  • Canon EOS M5
  • Canon PowerShot G16
  • Canon PowerShot G3 X
  • Canon PowerShot G7 X Mark II
  • Canon PowerShot G9 X Mark II
  • Fujifilm X-M1
  • Fujifilm X-Pro1
  • Fujifilm X-Pro2
  • Fujifilm X-T20
  • Leica X2
  • Nikon Coolpix A
  • Nikon D2X
  • Nikon D3000
  • Nikon D3400
  • Nikon D4
  • Nikon D500
  • Olympus E-M1MarkII
  • Olympus E-P5
  • Panasonic DMC-FZ200
  • Panasonic DMC-FZ300
  • Panasonic DMC-G7
  • Panasonic DMC-G70
  • Panasonic DMC-G8
  • Panasonic DMC-G80
  • Panasonic DMC-G81
  • Panasonic DMC-G85
  • Panasonic DMC-GX80
  • Panasonic DMC-GX85
  • Panasonic DMC-LX100
  • Panasonic DMC-TZ100
  • Panasonic DMC-TZ101
  • Panasonic DMC-TZ110
  • Panasonic DMC-ZS110
  • Pentax K-70
  • Sony DSC-RX100M5
  • Sony ILCA-68
  • Sony ILCE-5000
  • Sony ILCE-6500
  • Sony ILCE-7RM3

Translations

  • Catalan
  • Czech
  • Danish
  • Dutch
  • French
  • German
  • Greek
  • Hebrew
  • Hungarian
  • Italian
  • Japanese
  • Polish
  • Russian
  • Slovak
  • Slovenian
  • Spanish
  • Swedish
  • Ukrainian

December 21, 2017

a new website

A new year is coming on us quickly, so how about a nice new website to go with it?

Babynew Baby New Year from 110 years ago …

houz and I have been working hard over the past few months to migrate the old website from Wordpress to a new static site, using Python/Pelican. This should make things more secure and safer for both you and us (see the problems that rawsamples.ch had for the perils of using a db-driven backend for a website). Not to mention it makes collaboration and contributing a bit easier now, as the entire site gets its own GitHub repository (I’ll be eagerly awaiting your pull requests).

I tried to create a design that had a bit more emphasis on accessibility and readability. The site is fully responsive for screens ranging from a mobile phone to a full desktop. The type is larger and generally given a bit more room to breathe in the page and we tried to highlight images a bit more, as this is a photography-centered project.

This is just one small way for me to contribute and give back to the community in my own way (you should probably not let me near any real code). In fact, this is one of the reasons I started up the community over at PIXLS.US. We needed a community focused specifically on Free Software photography and a way for us to freely share our knowledge and experiences to help everyone, especially across multiple projects.

pixls.us

PIXLS.US Logo

If you’re not familiar with the community yet, why not?! We’re all photography folks who have a passion for Free Software with the mission:

To provide tutorials, workflows and a showcase for high-quality photography using Free/Open Source Software.

We also happen to have quite a few developers of various types in the community, and as a way to assist projects and contribute back we’ve been working on websites and other community-oriented functionality (like the forums, hosting comments, files, and more). I’d get in trouble if I didn’t mention that we also got raw.pixls.us setup to replace the ailing rawsamples.ch also.

One neat way we’re able to help out even more is by integrating the commenting system here into the forums. It’s how we manage commenting on the main pixls.us website, and we had great success with this approach for the digiKam project when we built them a new website earlier this year. This lets us moderate comments in one place, allows for cross-pollination of knowledge between the projects, and users get to truly own their comments (instead of being monetized and tracked by some third-party commenting system like Disqus).

Happy New Year

Have a look around the new site and please don’t hesitate to point out any problems you may run into! From me, thank you x1000 to the developers for this awesome project.

patdavid ♥’s darktable

December 20, 2017

Strings Freeze For GIMP 2.10 Is Now On

GIMP’s user interface is currently available in 80 languages. So far ca. 20 translations have been updated in the unstable branch since the beginning of the work on v2.10, and only 8 translations in the ‘po’ directory (where most translatable messages reside) are at least 90% complete. So clearly we need to give our translators a head start.

This is why GIMP’s master branch is now entering a tentative strings freeze phase in preparation for 2.10 release. We expect further changes between today and the v2.10 final release to affect no more than 1% of translatable messages. So it’s safe to start updating user interface translations now.

If you are interested, how complete the translation into your language is, check out the current stats. To start updating it, please contact your local team.

We would also like to remind translators that we are relaxing the release policy for the stable branch. Starting with v2.10, stable releases may have minor new features. This makes changes and the introduction of new strings more likely than in previous stable branches, where the only changes allowed were bug fixes (and which introduced only few string changes).

December 19, 2017

FreeCAD Arch Development News - November/December 2017

Hi all, First off, sorry for the delay, I was trying to finish the new Render workbench (see below) before posting this report, because otherwise there is not much exciting stuff this month, and it ended up taking more time than I thought, because I kept experimenting a lot until arriving to a good solution. And...

December 18, 2017

darktable 2.4.0rc2 released

we’re proud to announce the third release candidate for the upcoming 2.4 series of darktable, 2.4.0rc2!

the github release is here: https://github.com/darktable-org/darktable/releases/tag/release-2.4.0rc2.

as always, please don’t use the autogenerated tarball provided by github, but only our tar.xz. the checksum is:

$ sha256sum darktable-2.4.0rc2.tar.xz
dcb56e1eb2c10aa9fe64ea9ba3e806e3da3a3a0ebb47646a07e1838b88f15949 darktable-2.4.0rc1.tar.xz
$ sha256sum darktable-2.4.0rc2.dmg
5ad1c355c04d8a42bab7c2879cba92891dbdd0a89b8fe0ff2ea18f1f8b592f15 darktable-2.4.0rc1.dmg
$ sha256sum darktable-2.4.0rc2.dirty-win64.exe
a4cd63e9e44f029d4a85b430c5fdaf49e110c1ebe0a9cfc51ac2bf86ebac41cf darktable-2.4.0rc1.exe

Important note: to make sure that darktable can keep on supporting the raw file format for your camera, please read this post on how/what raw samples you can contribute to ensure that we have the full raw sample set for your camera under CC0 license!

changes since rc1

  • Fix a bug in haze removal that resulted in black areas in the exported image
  • Support Sony ILCE-7RM3
  • Make calls to exiv2’s readMetadata thread safe to not crash randomly
  • Don’t hide the mode combobox in the exposure module, just disable it
  • Change the styling of insensitive bauhaus widgets
  • Fix spurious segfault in local contrast
  • Don’t show an error popup on Windows when the CD drive is empty

and the changelog as compared to 2.2.0 can be found below. Some of the fixes might have been backported to the stable 2.2.x series already.

  • The maintainership of the RawSpeed library was transferred to the darktable project. The work on code cleanup, hardening, modernization, simplification and testing is ongoing.
  • Well over 2 thousand commits to darktable+rawspeed since 2.2.0
  • 244 pull requests handled
  • 320+ issues closed
  • Updated user manual is coming soon™

Hell Froze Over

  • As you might have read on our news post we finally ported darktable to Windows and intend to support it in the future. At the moment it’s still lacking a few features (for example there is not printing support), has a few limitations (tethering requires special drivers to be installed) and comes with its own set of bugs. But overall we are confident that it’s quite usable already and hope you will enjoy it. A very special thanks goes to Peter Budai who finally convinced us to agree to the port and who did most of the work.

The Big Ones

  • A new module for haze removal
  • The local contrast module can now be pushed much further, it also got a new local laplacian mode
  • Add undo support for masks and more intelligent grouping of undo steps
  • Blending now allows to display individual channels using false colors
  • darktable now supports loading Fujifilm compressed RAFs
  • darktable now supports loading floating point HDR DNGs as written by HDRMERGE
  • We also added channel specific blend modes for Lab and RGB color spaces
  • The base curve module allows for more control of the exposure fusion feature using the newly added bias slider
  • The tonecurve module now supports auto colour adjustment in RGB
  • Add absolute color input as an option to the color look up table module
  • A new X-Trans demosaicing algorithm, Frequency Domain Chroma, was implemented.
  • You can now choose from pre-defined scheduling profiles for OpenCL
  • Speaking of OpenCL, darktable now allows to force-use OpenCL for a specific pixelpipe
  • Xmp sidecar files are no longer written to disk when the content didn’t actually change. That mostly helps with network storage and backup systems that use files’ time stamps

New Features And Changes

  • Show a dialog window that tells when locking the database/library failed
  • Don’t shade the whole region on the map when searching for a location. Instead just draw a border around it.
  • Also in map mode: Clear the search list and map indicators when resetting the search module.
  • With OsmGPSMap newer than version 1.1.0 (i.e., anything released after that OsmGPSMap version) the map will show copyright info.
  • Running jobs with a progressbar (mostly import and export) will show that progress bar ontop the window entry in your task bar – if the system supports it. It should work on GNOME, KDE and Windows at least.
  • Add bash like string replacement for variables (export, watermark, session settings).
  • Add a preferences option to ask before removing empty dirs
  • The “colorbalance” module got a lot faster, thanks to SSE optimized code
  • Make gradient sliders a little more colorful
  • Make PNG compression level used for exporting configurable
  • On OSX, load single images from command line or via drag&drop in darkroom mode
  • Add an option to omit the intermediate tag hierarchy in exported files and only add the last level
  • In the watermark module, sort the list of SVG files and omit the file extension
  • Support XYZ as a proofing profile
  • Local contrast now got a new slider to set the midtone range
  • darktable got two new helper scripts (those are not installed by default, grab them from the sources): One to purge thumbnails that no longer have an associated image in the database, and a second script that uses inotify to watch a folder for new files to open them in a running darktable instance.
  • In the curve editors of base curve and tone curve you can now delete nodes with a right click and see coordinates of nodes while editing. Note that you can use keyboard modifiers ctrl and shift to change the precision of your changes
  • Creating a new instance of a module can now be done with a quick click of the middle mouse button on the multi-instance icon
  • New darktable installations on computers with more than 8 Gb of memory will now by default use half of that per module
  • Several background colors and the brush color are now configurable in the CSS
  • Some new cameras can bump the ISO level to insane highs. We try to follow as good as we can by no longer limiting it to 51200 in the GUI
  • Base curve and the highlights module now support multiple instances and use blending and masks
  • Having the 1 key toggle between 1 and 0 stars wasn’t very popular with many people. You can disable that extra feature and have it behave like the other rating shortcuts now
  • You can decide if you want to be asked before resetting the history stacks of images from the lighttable
  • The grain module was slightly changed to have a more pleasing, photographic-paper like appearance
  • Using the color look up table module you can now convert your images to monochrome, honoring the Helmholtz-Kohlrausch effect
  • Some more small improvements were made
  • Support basic import of Lightroom 7 settings
  • Change the styling of insensitive bauhaus widgets
  • Don’t hide the mode combobox in the exposure module, just disable it

Bugfixes

  • Fix the problem with rating images by accident when moving the mouse while typing an image size in the export module
  • Fix several oddities in folder and tag mode of the collect module.
  • Print mode’s color profile settings no longer interact with the export module
  • Update the style lists when importing a style
  • Fix some bugs with multiple module instances used in a style
  • On OSX only the main window should be fullscreen, not the popups
  • Some speedups with VERY big libraries or having A LOT OF tags
  • Significantly speed up tagging many images
  • Fix searching locations using OpenStreetMap
  • Fix partial copies of large files in “import from camera”
  • Fix a crash in the import dialog when using Lua to add widgets there
  • Fix some false-positive warnings about another running darktable instance and it having locked the databases
  • No longer switch to the favourite modules group when duplicating one of its modules
  • Fix loading of XYZ files
  • Fix Lab export when the profile was set from the lighttable
  • Create tmp snapshot files with mode 0600 to stop other people looking at them
  • Fix several bugs with Wayland. However, there are still issues, so darktable will prefer XWayland
  • Google deprecated the Picasa Web API so it’s no longer possible to create G+ albums
  • Fix the default for sliders with target not being “red” in the channel mixer
  • Fix the removing of directories
  • Make the escape key cancel history dialogs
  • Block keyboard accels when editing camera controls
  • Properly delete XMP sidecars
  • Make sure that the rating set in darktable is used for the exported file, not something set inside the raw file
  • Don’t re-write all XMP files when detaching a tag
  • Sync XMPs when a tag is removed from the database
  • Sync XMPs after a tag is attached/detached via the Lua API
  • Bail out of darktable-cli when the XMP file is not readable
  • Show ratings on zoomable lighttable without a delay
  • Rely on CUPS color management when printing without configuring any color profile in darktable
  • Many more bugs got fixed
  • Fix spurious segfault in local contrast
  • Make calls to exiv2’s readMetadata thread safe to not crash randomly

Lua

  • darktable now uses Lua 5.3. The bundled copy got updated accordingly
  • Add dt.print_log. It’s like print_error but without the ERROR prefix
  • Reorder callback parameters for intermediate export image: add the actual image to the parameters of the event
  • Call lua post-import-image event synchronously
  • Add darktable.configuration.running_os to detect the OS darktable is running on
  • New widget type: section_label, adds a label which looks like a section change

Changed Dependencies

  • CMake 3.1 is now required.
  • In order to compile darktable you now need at least gcc-4.9+/clang-3.4+, and gcc-5.0+ is highly recommended.
  • ZLIB is now required for the DNG Deflate compressed raw support.
  • darktable now uses Lua 5.3

Camera support, compared to 2.2.0

Warning: support for Nikon NEF ‘lossy after split’ raws was unintentionally broken due to the lack of such samples. Please see this post for more details. If you have affected raws, please contribute samples!

Base Support

  • Canon EOS 200D
  • Canon EOS Kiss X9
  • Canon EOS Rebel SL2
  • Canon EOS 6D Mark II (sRaw1, sRaw2)
  • Canon EOS 77D
  • Canon EOS 9000D
  • Canon EOS 800D
  • Canon EOS Kiss X9i
  • Canon EOS Rebel T7i
  • Canon EOS M100
  • Canon EOS M5
  • Canon EOS M6
  • Canon PowerShot G9 X Mark II
  • Canon PowerShot SX40 HS (dng)
  • Fujifilm GFX 50S (compressed)
  • Fujifilm X-A3
  • Fujifilm X-E2S
  • Fujifilm X-E3 (compressed)
  • Fujifilm X-Pro2 (compressed)
  • Fujifilm X-T2 (compressed)
  • Fujifilm X-T20 (compressed)
  • Fujifilm X100F (compressed)
  • GITUP GIT2P (chdk-a, chdk-b)
  • Kodak EasyShare Z980
  • LG D855 (dng)
  • LG H815 (dng)
  • LG Nexus 5X (dng)
  • LG US996 (dng)
  • LG VS995 (dng)
  • Leica D-LUX (Typ 109) (4:3, 3:2, 16:9, 1:1)
  • Leica X2 (dng)
  • Nikon COOLPIX B700 (12bit-uncompressed)
  • Nikon D500 (14bit-uncompressed, 12bit-uncompressed)
  • Nikon D5600 (12bit-compressed, 12bit-uncompressed, 14bit-compressed, 14bit-uncompressed)
  • Nikon D7500 (12bit-compressed, 14bit-compressed)
  • Nikon D850 (12bit-compressed, 12bit-uncompressed, 14bit-compressed, 14bit-uncompressed)
  • Nikon LS-5000 (dng)
  • Nokia Lumia 1020 (dng)
  • Olympus E-M10 Mark III
  • Olympus E-M1MarkII
  • Olympus TG-5
  • Panasonic DC-FZ82 (4:3)
  • Panasonic DMC-FZ80 (4:3)
  • Panasonic DMC-FZ85 (4:3)
  • Panasonic DC-GH5 (4:3)
  • Panasonic DC-FZ91 (4:3)
  • Panasonic DC-FZ92 (4:3)
  • Panasonic DC-FZ93 (4:3)
  • Panasonic DC-TZ90 (4:3)
  • Panasonic DC-ZS70 (4:3)
  • Panasonic DMC-FZ330 (4:3)
  • Panasonic DMC-GF6 (16:9, 3:2, 1:1)
  • Panasonic DMC-TZ61 (4:3, 3:2, 1:1, 16:9)
  • Panasonic DMC-ZS40 (4:3, 3:2, 1:1, 16:9)
  • Panasonic DMC-TZ80 (4:3)
  • Panasonic DMC-TZ81 (4:3)
  • Panasonic DMC-TZ85 (4:3)
  • Panasonic DMC-ZS60 (4:3)
  • Pentax K-5 (dng)
  • Pentax K-r (dng)
  • Pentax K10D (dng)
  • Phase One IQ140
  • Samsung G920F
  • Samsung G935F
  • Samsung GX10
  • Sony ILCE-6500
  • Sony ILCE-7RM3
  • Sony ILCE-9

White Balance Presets

  • Canon EOS 6D Mark II
  • Fujifilm X-T20
  • Fujifilm X100F
  • Nikon 1 AW1
  • Nikon Coolpix A
  • Panasonic DMC-GX80
  • Panasonic DMC-GX85
  • Panasonic DMC-TZ100
  • Panasonic DMC-TZ101
  • Panasonic DMC-TZ110
  • Panasonic DMC-ZS110
  • Pentax K-3 II

Noise Profiles

  • Canon EOS 1300D
  • Canon EOS Kiss X80
  • Canon EOS Rebel T6
  • Canon EOS 5D Mark IV
  • Canon EOS 6D Mark II
  • Canon EOS M5
  • Canon PowerShot G16
  • Canon PowerShot G3 X
  • Canon PowerShot G7 X Mark II
  • Canon PowerShot G9 X Mark II
  • Fujifilm X-M1
  • Fujifilm X-Pro1
  • Fujifilm X-T20
  • Leica X2
  • Nikon Coolpix A
  • Nikon D2X
  • Nikon D3000
  • Nikon D3400
  • Nikon D4
  • Nikon D500
  • Olympus E-M1MarkII
  • Olympus E-P5
  • Panasonic DMC-FZ200
  • Panasonic DMC-FZ300
  • Panasonic DMC-G7
  • Panasonic DMC-G70
  • Panasonic DMC-G8
  • Panasonic DMC-G80
  • Panasonic DMC-G81
  • Panasonic DMC-G85
  • Panasonic DMC-GX80
  • Panasonic DMC-GX85
  • Panasonic DMC-LX100
  • Panasonic DMC-TZ100
  • Panasonic DMC-TZ101
  • Panasonic DMC-TZ110
  • Panasonic DMC-ZS110
  • Pentax K-70
  • Sony DSC-RX100M5
  • Sony ILCA-68
  • Sony ILCE-5000
  • Sony ILCE-6500

Updated Translations

  • Catalan
  • Dutch
  • French
  • German
  • Hebrew
  • Hungarian
  • Polish
  • Russian
  • Spanish

December 16, 2017

Homemade Arduino Part 2: a Bare Atmega328 Without a Clock

Playing with the ATtiny85 I was struck by how simple the circuit was. Sure, I'd made a homemade Arduino on a breadboard; but with the crystal and all the extra capacitors and resistors it ends up seeming like a lot of parts and wires. If an ATtiny can use a built-in clock and not need all those extra parts, couldn't I use an Atmega328 the same way?

[Circuit for Atmega328 on breadboard with ISP] Why, yes, as it turns out. But there are a few tricks.

Wire it

Wiring a bare Atmega chip is easy. You'll want to keep a good pinout diagram handy, like this Arduino ATmega328 Pinout from HobbyTronics.

For the initial wiring, all you need is two power and two ground lines, the pins marked - and +, plus a pullup resistor on RST (something large, like 10kΩ). The excellent tutorial From Arduino to a Microcontroller on a Breadboard is a good guide if you need additional details: the third section shows a circuit without external clock.

Add an LED and resistor on pin 13 (atmega pin 19, called SCK) so you can test it using a blink program.

Now you need to set up the software.

Set up a hardware profile for a bare Arduino

To program it with the Arduino libraries, you'll need a hardware definition for an atmega328 chip with an internal clock. I used the download from the last section of the excellent tutorial, From Arduino to a Microcontroller on a Breadboard. (Keep that page up: it has good wiring diagrams.)

For Arduino 1.8.5, download breadboard-1-6-x.zip and unpack it in your ~/sketchbook/hardware/ directory, making a directory there called breadboard. Then you'll need to make one change: the 1.6 directory is missing a file called pins_arduino.h", so if you try to compile with this hardware definition, you'll get an error like:

mkdir -p build-atmega328bb-atmega328
/usr/local/share/arduino/hardware/tools/avr/bin/avr-g++ -x c++ -include Arduino.h -MMD -c -mmcu=atmega328p -DF_CPU=8000000L -DARDUINO=185 -DARDUINO_ARCH_AVR -D__PROG_TYPES_COMPAT__ -I/usr/local/share/arduino/hardware/arduino/avr/cores/arduino -I/home/akkana/sketchbook/hardware/breadboard/avr/variants/standard    -Wall -ffunction-sections -fdata-sections -Os -fpermissive -fno-exceptions -std=gnu++11 -fno-threadsafe-statics -flto blink.ino -o build-atmega328bb-atmega328/blink.ino.o
In file included from :0:0:
/usr/local/share/arduino/hardware/arduino/avr/cores/arduino/Arduino.h:257:26: fatal error: pins_arduino.h: No such file or directory
 #include "pins_arduino.h"
                          ^
compilation terminated.
/usr/share/arduino/Arduino.mk:1251: recipe for target 'build-atmega328bb-atmega328/blink.ino.o' failed
make: *** [build-atmega328bb-atmega328/blink.ino.o] Error 1

The problem is that it's including these directories:
-I/usr/local/share/arduino/hardware/arduino/avr/cores/arduino -I/home/akkana/sketchbook/hardware/breadboard/avr/variants/standard
but the actual file is in:
/usr/local/share/arduino/hardware/arduino/avr/variants/standard/pins_arduino.h

You can fix that by making a link from the "standard" directory in your Arduino install to breadboard/avr/variants/standard. On Linux, that would be something like this (Mac and Windows people can substitute their local equivalents):

ln -s /usr/local/share/arduino/hardware/arduino/avr/variants/standard ~/sketchbook/hardware/breadboard/avr/variants/

Now your hardware definition should be ready to go. To check, fire up the IDE and look in Tools->Board for ATmega328 on a breadboard (8 MHz internal clock). Or if you use Arduino-mk, run ALTERNATE_CORE=breadboard make show_boards and make sure it lists atmega328bb ATmega328 on a breadboard (8 MHz internal clock).

Reprogram the Fuses and Bootloader for an Internal Clock

The next trick is that an Atmega chip programmed with the Arduino bootloader is also fused to use an external, 16MHz clock. If you wire it to use its internal 8MHz clock, you won't be able to talk to it with either an ISP or FTDI.

You'll definitely run into this if you pull the CPU out of an Arduino. But even if you buy new chips you may see it: many Atmega328s come pre-programmed with the Arduino bootloader. After all, that's what most people want.

The easiest way to reprogram the fuses is to use the hardware definition you just installed to burn a new bootloader, which resets the fuse settings at the same time. So you need an In-System Programmer, or ISP. You can use an Arduino as an ISP, but I'm told that this tends to be flaky and isn't recommended. After I had problems using an Arduino I ordered a cheap USBtinyUSP, which works fine.

Regardless of which ISP you use, if you wire up your atmega without an external clock when it's fused for one, you won't be able to burn a bootloader. A typical error:

[ ... ]
Reading | ################################################## | 100% 0.02s

avrdude: Device signature = 0x000000 (retrying)

Error while burning bootloader.
Reading | ################################################## | 100% 0.02s

avrdude: Device signature = 0x000000
avrdude: Yikes!  Invalid device signature.
     Double check connections and try again, or use -F to override
     this check.

The solution is to burn the bootloader using an external clock. You can add a crystal and two capacitors to your breadboard circuit if you have them. If not, an easy solution is to pull the chip out of the breadboard, plug it into the socket in an Arduino and burn it there. (Note: if you're using an Arduino as your ISP, you'll need a second Arduino.)

Plug your ISP into the Arduino's ISP header: on an Uno, that's the header labeled ICSP at the end of the chip farthest away from the USB plug. It's a six-pin connector (2x3), it's easy to plug in backward and you can't depend on either the Arduino's header or the ISP's cable being labeled as to direction; if in doubt, use a multimeter in continuity mode to see which pin is ground on each side, then make sure those pins match. Once you're sure, mark your connector somehow so you'll know next time.

In the Arduino IDE, set Tools->Board to ATmega328 on a breadboard (8 MHz internal clock), set Programmer to whatever ISP you're using. then run Tools->Burn Bootloader.

If you're using Arduino-mk instead of the IDE, set up a Makefile that looks like this:

ALTERNATE_CORE = breadboard
BOARD_TAG      = atmega328bb
ISP_PROG     = usbtiny
include /usr/local/share/Arduino-Makefile/Arduino.mk
Substitute your ISP, if different, and your location for Arduino.mk. Then type make burn_bootloader

Program it

Once you're wired, you should be able to program it either with an FTDI board or an ISP, as I discussed in homemade Arduino, Part 1. You should be able to use your minimal Atmega328 to run anything you can run on a normal Arduino (albeit at half the clock speed).

I plan to make a little board with a ZIF socket and connectors for both the USBtinyISP and the FTDI Friend so I don't have to plug in all those wires again each time.

December 15, 2017

More Bluetooth (and gaming) features

In the midst of post-release bug fixing, we've also added a fair number of new features to our stack. As usual, new features span a number of different components, so integrators will have to be careful picking up all the components when, well, integrating.

PS3 clones joypads support

Do you have a PlayStation 3 joypad that feels just a little bit "off"? You can't find the Sony logo anywhere on it? The figures on the face buttons look like barbed wire? And if it were a YouTube video, it would say "No copyright intended"?


Bingo. When plugged in via USB, those devices advertise themselves as SHANWAN or Gasia, and implement the bare minimum to work when plugged into a PlayStation 3 console. But as a Linux computer would behave slightly differently, we need to fix a couple of things.

The first fix was simple, but necessary to be able to do any work: disable the rumble motor that starts as soon as you plug the pad through USB.

Once that's done, we could work around the fact that the device isn't Bluetooth compliant, and hard-code the HID service it's supposed to offer.

Bluetooth LE Battery reporting

Bluetooth Low Energy is the new-fangled (7-year old) protocol for low throughput devices, from a single coin-cell powered sensor, to input devices. What's great is that there's finally a standardised way for devices to export their battery statuses. I've added support for this in BlueZ, which UPower then picks up for desktop integration goodness.

There are a number of Bluetooth LE joypads available for pickup, including a few that should be firmware upgradeable. Look for "Bluetooth 4" as well as "Bluetooth LE" when doing your holiday shopping.

gnome-bluetooth work

Finally, this is the boring part. Benjamin and I reworked code that's internal to gnome-bluetooth, as used in the Settings panel as well as the Shell, to make it use modern facilities like GDBusObjectManager. The overall effect of this is, less code, less brittle and more reactive when Bluetooth adapters come and go, such as when using airplane mode.

Apart from the kernel patch mentioned above (you'll know if you need it :), those features have been integrated in UPower 0.99.7 and in the upcoming BlueZ 5.48. And they will of course be available in Fedora, both in rawhide and as updates to Fedora 27 as soon as the releases have been done and built.

GG!

December 12, 2017

LGM and Libre Graphics at SCaLE 16x


LGM and Libre Graphics at SCaLE 16x

All the libre graphics!

There are two libre graphics related meetings coming up early next year. The annual Libre Graphics Meeting (in Spain this year), and something entirely new: a libre graphics track at SCaLE. How exciting!

Libre Graphics Meeting 2018

LGM Logo SVG

The Libre Graphics Meeting is going to be in Seville, Spain this year. They recently published their Call for Participation and are accepting presentation and talk proposals now. Unfortunately, I won’t be able to attend this year, but there’s a pretty good chance some friendlier folks from the community will be! We’ll update more about who will be making it out as soon as we know, and maybe we can convince someone to run another photowalk with everyone. (On a side note, if anyone from the community is going to make it and wants a hand putting anything together for a presentation just let us know - we’re here to help.)

Libre Graphics at SCaLE (California, USA)

SCaLE 16x Logo

This year we have a neat announcement - due to some prodding from Nate Willis, we have been given a day at the Southern California Linux Expo (SCaLE) to hold a Libre Graphics focused track! The expo is at the Pasadena Convention Center, March 8-11, 2018.

We first had a chance to hang out with LWN editor Nate Willis during the Libre Graphics Meeting 2016 in London, and later out at the Texas Linux Fest. GIMP was able to have both Akkana Peck and myself out to present on GIMPy stuff and host a photowalk as well.

The organizer for SCaLE, Ilan, was kind enough to give us a day (Friday, March 9th) and a room for all the libre graphics artists, designers, programmers, and hackers.

You could come meet the face behind these avatars.

I will be in attendance promoting GIMP stuff in the main track, Dr. Ullah (Isaac Ullah) will hopefully be presenting, and Mica will be there (@paperdigits) as well. I’m pretty certain we’ll be holding a photowalk for attendees while we’re there - and we may even setup a nice headshot booth in the expo to take free headshots for folks.

We would love to see some folks out there. If you think you might be able to make it, or even better submit a talk proposal, please come and join us! (I was thinking about getting an AirBnB to stay in, so if folks let me know they are going to make it out we can coordinate a place to all stay together.)

SCaLE Libre Graphics Track Call for Participation

The libre graphics community is thrilled to announce that a special, one-day track at SCaLE 16x will be dedicated to libre graphics software and artists. All those who work with free and open-source tools for creative graphics projects are invited to submit a proposal and join us for the day!

SCaLE 16x will take place from March 8 to 11 of 2018 in Pasadena California. Libre Graphics Day: SCaLE will take place at the main SCaLE venue on Friday, March 9.

The libre graphics track is an opportunity for teams, contributors and practitioners involved in Libre Graphics projects to share their experiences, showcase new developments, and hear new and inspiring ideas.

By libre graphics we mean “free, Libre and Open Source tools for creative uses”. Libre graphics is not just about software, but extends to standards and file formats used in creative work.

People from around the world who are passionate about Free/Libre tools and their creative applications are encouraged to submit a talk proposal. Sessions will be 30 minutes in length.

Developers, artists, and activists alike are invited. First-time presenters and established projects of all sizes are welcome to submit.

We are looking for:

  • Reflections and practical sessions on promoting the philosophy and use of Libre Graphics tools.
  • Technical presentations and workshops for developers.
  • Showcases of excellent work made using Libre Graphics tools.
  • New tools and workflows for graphics and code.
  • Reflections on the activities of existing Free/Libre and Open Source communities.

Submit

Please submit your proposal to graphics-cfp@socallinuxexpo.org.

If you have any questions feel free to reach out to me on the forum.

Deadline

The deadline for submissions is January 10th, 2018, and participants will be notified by the end of January 2018.

Namaste ! (on the road to Swatantra 2017)

This is a little blog post from India. I’ve been invited to give not one, but two talks at Swatantra 2017, the triennial conference organised by ICFOSS in Thiruvananthapuram (also known by its shorter old name, Trivandrum), Kerala.

I’ll have the pleasure to give a talk about GCompris, and another one about Synfig studio. It’s been a long time since I didn’t talk about the latter, but since Konstantin Dmitriev and the Morevna team were not available, I’ll do my best to represent Synfig there.





(little teaser animation of the event banner, done with Synfig studio)

I’ll also meet some friends from Krita, David Revoy and Raghavendra Kamath, so even if there is no talk dedicated to Krita, it should be well represented.

The event will happen the 20th and 21st of December, and my talks will be on the second day. Until then, I’m spending a week visiting and enjoying the south of India.

You can find more info on the official website of the event: swatantra.net.in. Many thanks again to the nice organization team at ICFOSS for the invitation !

darktable 2.4.0rc1 released

we’re proud to announce the second release candidate for the upcoming 2.4 series of darktable, 2.4.0rc1!

the github release is here: https://github.com/darktable-org/darktable/releases/tag/release-2.4.0rc1.

as always, please don’t use the autogenerated tarball provided by github, but only our tar.xz. the checksum is:

$ sha256sum darktable-2.4.0rc1.tar.xz
2b38462584223a0f74f081dc025e1811b524f403d919734a1b8c15f7c87858ea darktable-2.4.0rc1.tar.xz
$ sha256sum darktable-2.4.0rc1.dmg
??? darktable-2.4.0rc1.dmg
$ sha256sum darktable-2.4.0rc1.exe
d576071f7052d61acf35d05184d5e12c2bdedcb1dce0159668022c2e46c6467d darktable-2.4.0rc1.exe

Important note: to make sure that darktable can keep on supporting the raw file format for your camera, please read this post on how/what raw samples you can contribute to ensure that we have the full raw sample set for your camera under CC0 license!

changes since rc0

  • noise profile for Nikon D4
  • Phase One IQ140 support
  • OSX packaging fixes
  • Lightroom 7 import fixes
  • Some fixes for sliders and comboboxen and grabbing the keyboard focus
  • No longer use colored sliders in the white balance module – they confused people
  • Update Catalan translation
  • Update Hungarian translation
  • Fix OpenCL on OSX
  • Bail out of darktable-cli when the XMP file is not readable
  • Fix timezone selection for geotagging on Windows
  • Canon EOS M100 supported
  • Show ratings on zoomable lighttable without a delay
  • Rely on CUPS color management when printing without configuring any color profile in darktable

and the changelog as compared to 2.2.0 can be found below. Some of the fixes might have been backported to the stable 2.2.x series already.

  • The maintainership of the RawSpeed library was transferred to the darktable project. The work on code cleanup, hardening, modernization, simplification and testing is ongoing.
  • Well over 2 thousand commits to darktable+rawspeed since 2.2.0
  • 244 pull requests handled
  • 320+ issues closed
  • Updated user manual is coming soon™

Hell Froze Over

  • As you might have read on our news post we finally ported darktable to Windows and intend to support it in the future. At the moment it’s still lacking a few features (for example there is not printing support), has a few limitations (tethering requires special drivers to be installed) and comes with its own set of bugs. But overall we are confident that it’s quite usable already and hope you will enjoy it. A very special thanks goes to Peter Budai who finally convinced us to agree to the port and who did most of the work.

The Big Ones

  • A new module for haze removal
  • The local contrast module can now be pushed much further, it also got a new local laplacian mode
  • Add undo support for masks and more intelligent grouping of undo steps
  • Blending now allows to display individual channels using false colors
  • darktable now supports loading Fujifilm compressed RAFs
  • darktable now supports loading floating point HDR DNGs as written by HDRMERGE
  • We also added channel specific blend modes for Lab and RGB color spaces
  • The base curve module allows for more control of the exposure fusion feature using the newly added bias slider
  • The tonecurve module now supports auto colour adjustment in RGB
  • Add absolute color input as an option to the color look up table module
  • A new X-Trans demosaicing algorithm, Frequency Domain Chroma, was implemented.
  • You can now choose from pre-defined scheduling profiles for OpenCL
  • Speaking of OpenCL, darktable now allows to force-use OpenCL for a specific pixelpipe
  • Xmp sidecar files are no longer written to disk when the content didn’t actually change. That mostly helps with network storage and backup systems that use files’ time stamps

New Features And Changes

  • Show a dialog window that tells when locking the database/library failed
  • Don’t shade the whole region on the map when searching for a location. Instead just draw a border around it.
  • Also in map mode: Clear the search list and map indicators when resetting the search module.
  • With OsmGPSMap newer than version 1.1.0 (i.e., anything released after that OsmGPSMap version) the map will show copyright info.
  • Running jobs with a progressbar (mostly import and export) will show that progress bar ontop the window entry in your task bar – if the system supports it. It should work on GNOME, KDE and Windows at least.
  • Add bash like string replacement for variables (export, watermark, session settings).
  • Add a preferences option to ask before removing empty dirs
  • The “colorbalance” module got a lot faster, thanks to SSE optimized code
  • Make gradient sliders a little more colorful
  • Make PNG compression level used for exporting configurable
  • On OSX, load single images from command line or via drag&drop in darkroom mode
  • Add an option to omit the intermediate tag hierarchy in exported files and only add the last level
  • In the watermark module, sort the list of SVG files and omit the file extension
  • Support XYZ as a proofing profile
  • Local contrast now got a new slider to set the midtone range
  • darktable got two new helper scripts (those are not installed by default, grab them from the sources): One to purge thumbnails that no longer have an associated image in the database, and a second script that uses inotify to watch a folder for new files to open them in a running darktable instance.
  • In the curve editors of base curve and tone curve you can now delete nodes with a right click and see coordinates of nodes while editing. Note that you can use keyboard modifiers ctrl and shift to change the precision of your changes
  • Creating a new instance of a module can now be done with a quick click of the middle mouse button on the multi-instance icon
  • New darktable installations on computers with more than 8 Gb of memory will now by default use half of that per module
  • Several background colors and the brush color are now configurable in the CSS
  • Some new cameras can bump the ISO level to insane highs. We try to follow as good as we can by no longer limiting it to 51200 in the GUI
  • Base curve and the highlights module now support multiple instances and use blending and masks
  • Having the 1 key toggle between 1 and 0 stars wasn’t very popular with many people. You can disable that extra feature and have it behave like the other rating shortcuts now
  • You can decide if you want to be asked before resetting the history stacks of images from the lighttable
  • The grain module was slightly changed to have a more pleasing, photographic-paper like appearance
  • Using the color look up table module you can now convert your images to monochrome, honoring the Helmholtz-Kohlrausch effect
  • Some more small improvements were made
  • Support basic import of Lightroom 7 settings

Bugfixes

  • Fix the problem with rating images by accident when moving the mouse while typing an image size in the export module
  • Fix several oddities in folder and tag mode of the collect module.
  • Print mode’s color profile settings no longer interact with the export module
  • Update the style lists when importing a style
  • Fix some bugs with multiple module instances used in a style
  • On OSX only the main window should be fullscreen, not the popups
  • Some speedups with VERY big libraries or having A LOT OF tags
  • Significantly speed up tagging many images
  • Fix searching locations using OpenStreetMap
  • Fix partial copies of large files in “import from camera”
  • Fix a crash in the import dialog when using Lua to add widgets there
  • Fix some false-positive warnings about another running darktable instance and it having locked the databases
  • No longer switch to the favourite modules group when duplicating one of its modules
  • Fix loading of XYZ files
  • Fix Lab export when the profile was set from the lighttable
  • Create tmp snapshot files with mode 0600 to stop other people looking at them
  • Fix several bugs with Wayland. However, there are still issues, so darktable will prefer XWayland
  • Google deprecated the Picasa Web API so it’s no longer possible to create G+ albums
  • Fix the default for sliders with target not being “red” in the channel mixer
  • Fix the removing of directories
  • Make the escape key cancel history dialogs
  • Block keyboard accels when editing camera controls
  • Properly delete XMP sidecars
  • Make sure that the rating set in darktable is used for the exported file, not something set inside the raw file
  • Don’t re-write all XMP files when detaching a tag
  • Sync XMPs when a tag is removed from the database
  • Sync XMPs after a tag is attached/detached via the Lua API
  • Bail out of darktable-cli when the XMP file is not readable
  • Show ratings on zoomable lighttable without a delay
  • Rely on CUPS color management when printing without configuring any color profile in darktable
  • Many more bugs got fixed

Lua

  • darktable now uses Lua 5.3. The bundled copy got updated accordingly
  • Add dt.print_log. It’s like print_error but without the ERROR prefix
  • Reorder callback parameters for intermediate export image: add the actual image to the parameters of the event
  • Call lua post-import-image event synchronously
  • Add darktable.configuration.running_os to detect the OS darktable is running on
  • New widget type: section_label, adds a label which looks like a section change

Changed Dependencies

  • CMake 3.1 is now required.
  • In order to compile darktable you now need at least gcc-4.9+/clang-3.4+, and gcc-5.0+ is highly recommended.
  • ZLIB is now required for the DNG Deflate compressed raw support.
  • darktable now uses Lua 5.3

Camera support, compared to 2.2.0

Warning: support for Nikon NEF ‘lossy after split’ raws was unintentionally broken due to the lack of such samples. Please see this post for more details. If you have affected raws, please contribute samples!

Base Support

  • Canon EOS 200D
  • Canon EOS Kiss X9
  • Canon EOS Rebel SL2
  • Canon EOS 6D Mark II (sRaw1, sRaw2)
  • Canon EOS 77D
  • Canon EOS 9000D
  • Canon EOS 800D
  • Canon EOS Kiss X9i
  • Canon EOS Rebel T7i
  • Canon EOS M100
  • Canon EOS M5
  • Canon EOS M6
  • Canon PowerShot G9 X Mark II
  • Canon PowerShot SX40 HS (dng)
  • Fujifilm GFX 50S (compressed)
  • Fujifilm X-A3
  • Fujifilm X-E2S
  • Fujifilm X-E3 (compressed)
  • Fujifilm X-Pro2 (compressed)
  • Fujifilm X-T2 (compressed)
  • Fujifilm X-T20 (compressed)
  • Fujifilm X100F (compressed)
  • GITUP GIT2P (chdk-a, chdk-b)
  • Kodak EasyShare Z980
  • LG D855 (dng)
  • LG H815 (dng)
  • LG Nexus 5X (dng)
  • LG US996 (dng)
  • LG VS995 (dng)
  • Leica D-LUX (Typ 109) (4:3, 3:2, 16:9, 1:1)
  • Leica X2 (dng)
  • Nikon COOLPIX B700 (12bit-uncompressed)
  • Nikon D500 (14bit-uncompressed, 12bit-uncompressed)
  • Nikon D5600 (12bit-compressed, 12bit-uncompressed, 14bit-compressed, 14bit-uncompressed)
  • Nikon D7500 (12bit-compressed, 14bit-compressed)
  • Nikon D850 (12bit-compressed, 12bit-uncompressed, 14bit-compressed, 14bit-uncompressed)
  • Nikon LS-5000 (dng)
  • Nokia Lumia 1020 (dng)
  • Olympus E-M10 Mark III
  • Olympus E-M1MarkII
  • Olympus TG-5
  • Panasonic DC-FZ82 (4:3)
  • Panasonic DMC-FZ80 (4:3)
  • Panasonic DMC-FZ85 (4:3)
  • Panasonic DC-GH5 (4:3)
  • Panasonic DC-FZ91 (4:3)
  • Panasonic DC-FZ92 (4:3)
  • Panasonic DC-FZ93 (4:3)
  • Panasonic DC-TZ90 (4:3)
  • Panasonic DC-ZS70 (4:3)
  • Panasonic DMC-FZ330 (4:3)
  • Panasonic DMC-GF6 (16:9, 3:2, 1:1)
  • Panasonic DMC-TZ61 (4:3, 3:2, 1:1, 16:9)
  • Panasonic DMC-ZS40 (4:3, 3:2, 1:1, 16:9)
  • Panasonic DMC-TZ80 (4:3)
  • Panasonic DMC-TZ81 (4:3)
  • Panasonic DMC-TZ85 (4:3)
  • Panasonic DMC-ZS60 (4:3)
  • Pentax K-5 (dng)
  • Pentax K-r (dng)
  • Pentax K10D (dng)
  • Phase One IQ140
  • Samsung G920F
  • Samsung G935F
  • Samsung GX10
  • Sony ILCE-6500
  • Sony ILCE-9

White Balance Presets

  • Canon EOS 6D Mark II
  • Fujifilm X-T20
  • Fujifilm X100F
  • Nikon 1 AW1
  • Nikon Coolpix A
  • Panasonic DMC-GX80
  • Panasonic DMC-GX85
  • Panasonic DMC-TZ100
  • Panasonic DMC-TZ101
  • Panasonic DMC-TZ110
  • Panasonic DMC-ZS110
  • Pentax K-3 II

Noise Profiles

  • Canon EOS 1300D
  • Canon EOS Kiss X80
  • Canon EOS Rebel T6
  • Canon EOS 5D Mark IV
  • Canon EOS 6D Mark II
  • Canon EOS M5
  • Canon PowerShot G16
  • Canon PowerShot G3 X
  • Canon PowerShot G7 X Mark II
  • Canon PowerShot G9 X Mark II
  • Fujifilm X-M1
  • Fujifilm X-Pro1
  • Fujifilm X-T20
  • Leica X2
  • Nikon Coolpix A
  • Nikon D2X
  • Nikon D3000
  • Nikon D3400
  • Nikon D4
  • Nikon D500
  • Olympus E-M1MarkII
  • Olympus E-P5
  • Panasonic DMC-FZ200
  • Panasonic DMC-FZ300
  • Panasonic DMC-G7
  • Panasonic DMC-G70
  • Panasonic DMC-G8
  • Panasonic DMC-G80
  • Panasonic DMC-G81
  • Panasonic DMC-G85
  • Panasonic DMC-GX80
  • Panasonic DMC-GX85
  • Panasonic DMC-LX100
  • Panasonic DMC-TZ100
  • Panasonic DMC-TZ101
  • Panasonic DMC-TZ110
  • Panasonic DMC-ZS110
  • Pentax K-70
  • Sony DSC-RX100M5
  • Sony ILCA-68
  • Sony ILCE-5000
  • Sony ILCE-6500

Updated Translations

  • Catalan
  • Dutch
  • French
  • German
  • Hebrew
  • Hungarian
  • Russian
  • Spanish

December 11, 2017

GIMP 2.9.8 Released

Newly released GIMP 2.9.8 introduces on-canvas gradient editing and various enhancements while focusing on bugfixing and stability. For a complete list of changes please see NEWS.

On-Canvas Gradient Editing

One of the most user-visible changes in 2.9.8 is the updated Blend tool. Here’s what’s new about it.

First of all, it pretty much eliminates the need for the old Gradient Editor dialog, as all of the dialog’s features are now available directly on the canvas. You can create and delete color stops, select and shift them, assign colors to color stops, change blending and coloring for segments between color stops, create new color stops from midpoints.

Secondly, default gradients are now “editable”. As you probably know, the reason most resources such as brushes, painting dynamics, and gradients are not direclty editable is that they are typically installed into a system directory where non-privileged user can’t make any changes.

Now when you try to change an existing gradient from a system folder, GIMP will create a copy of it, call it a Custom Gradient and preserve it across sessions. Unless, of course, you edit another ‘system’ gradient, in which case it will become the new custom gradient.

Since this feature is useful for more than just gradients, it was made generic enough to be used for brushes and other types of resources in the future. We expect to revisit this in the future releases of GIMP.

Now that 2.9.8 is out with the updated Blend tool, we are interested in your feedback, as we still expect some cleanup and enhancements to be done there.

Most of the programming was done by Ell, however we also want to acknowledge two other people who contributed to that effort one way or another.

Michael Henning improved the Blend tool for 2.9.2, making the position of its endpoints editable before applying the gradient fill.

Michael Natterer refactored source code of GIMP’s tools to make them reuse one another’s on-canvas handles. That greatly simplified adding on-canvas handles for color stops. He also added the generic on-canvas dialog with the most important options for tools.

Clip Warning

Ell also implemented a feature request made in our public mailing list, where Elle Stone asked for some way to visualize underexposed and overexposed areas of a photo, which is a common feature in digital photography tools such as darktable and RawTherapee.

The new Clip Warning display filter targets that use case and fills underexposed and overexposed areas with user-configurable colors. For now, it’s mostly geared towards images where colors are stored with floating point precision. You will mostly benefit from this, if you work on 16/32 bit per channel float images such as EXR and TIFF.

Implementing this feature as a display filter has certain disadvantages such as having to go through the whole routine of adding a display filter for every image. We are thinking of better ways to do this.

Color Management

GIMP now uses the babl library for doing conversion of images between color spaces when matrix-based ICC profiles are used. This leads to completing transforms ca. 5 times faster in comparison to LittleCMS v2 on a few test images we tried this on. We expect to make further use of babl for doing color transforms once the library supports ICC profiles based on lookup tables.

Wayland support

While we already had the screenshot plug-in working under GNOME/Wayland, we now implemented screenshots for KDE/Wayland (though it misses rectangular area selection).

The Color Picker widget will now also work in KDE/Wayland. Note that there is still no color-picking interface in GNOME for Wayland, so as a workaround, color picking will only work inside GIMP windows for this platform.

Color-picked and screenshot pixels are not color-managed yet in Wayland.

Paste in Place

Michael Natterer implemented another small feature request from a user who asked for an Inkscape-like Paste in Place command. The idea is that GIMP should be able to paste contents of the clipboard at exact coordinates the contents was originally copied from. This feature is available for both the regular clipboard and named buffers.

Paste in Place complements the usual Paste command which places contents of the clipboard into the center of the viewport.

GUI and Usability

The spinscale widget now highlights vertical parts of the slider section differently to hint that position of cursor above the widget matters. When changing values in the lower step section, the pointer will be wrapped around the screen so that you could continue adjusting the value without interruptions.

spinscale widget highlighting lower area

When using transform tools, you can now press a modifier key before or after pressing/releasing a mouse button.

The Info window for the color picker now remembers the modes across session. So if you prefer seeing LAB values, that’s what you will see every time until you choose something else.

Canvas rotation and flip information is now visible in the status bar, as angle value and flip icon. Clicking on these canvas statuses will respectively raise the Select Rotation Angle dialog or unflip the canvas.

Status about canvas rotation and flip

Help Manuals

Upon detection of locally installed manuals in several languages, GIMP will now allow selection of the preferred manual language in the Preferences dialog (Interface > Help System).

manual localization in preferences Manual localization settings in GIMP’s preferences

This is especially useful since GIMP’s interface is available in 80 languages, while its manual is translated to only 17 languages. You may therefore not have a choice of viewing the manual in your preferred language.

Moreover, some people choose English over their native language for user interfaces, while sticking to their native language for reading documentation. This is another case where choosing preferred language for the user manual might come in handy.

Improved Wavelet Decompose Filter

The much demanded Wavelet Decompose filter got a small round of updates and gained a couple of new options: placing decomposition stack into its own layer group and adding a layer mask to each scales layers. It also produces more expected results now.

File Formats

The PSD plug-in was fixed to properly handle Photoshop files with deeply nested layer groups and preserve expanded state of groups for both importing and exporting. Additional changes fix mask position and improve layer opacity for importing/exporting.

The PDF plug-in now supports loading password-protected files by promting the user for password.

HGT files can now be imported. HGT is the format for Digital Elevation Model data by the NASA and other space agencies. GIMP now supports both the SRTM-1 and SRTM-3 types (as far as we know, the only two variants) which will be imported as grayscale RGB images.

Importing HGT files in GIMP NASA HGT file import followed by appropriate “Gradient Map” filtering

In order to obtain more visible relief information, you will want to map altitudes to colors, for instance with the “Gradient Map” filter as we did in the example image above (see also this explicative post on the process).

Translations

Translation of GIMP was updated for 13 languages: Catalan, Croatian, Galician, German, Greek, Hungarian, Icelandic, Indonesian, Italian, Polish, Russian, Spanish, and Swedish.

What’s Next

We’ll enter strings freeze soon so that translators could safely finalize their work for 2.10. Following that we expect to start making release candidates of GIMP 2.10.

CSR devices now supported in fwupd

On Friday I added support for yet another variant of DFU. This variant is called “driverless DFU” and is used only by BlueCore chips from Cambridge Silicon Radio (now owned by Qualcomm). The driverless just means that it’s DFU like, and routed over HID, but it’s otherwise an unremarkable protocol. CSR is a huge ODM that makes most of the Bluetooth audio chips in vendor hardware. The hardware vendor can enable or disable features on the CSR microcontroller depending on licensing options (for instance echo cancellation), and there’s even a little virtual machine to do simple vendor-specific things. All the CSR chips are updatable in-field, and most vendors issue updates to fix sound quality issues or to add support for new protocols or devices.

The BlueCore CSR chips are used everywhere. If you have a “wireless” speaker or headphones that uses Bluetooth there is a high probability that it’s using a CSR chip inside. This makes the addition of CSR support into fwupd a big deal to access a lot of vendors. It’s a lot easier to say “just upload firmware” rather than “you have to write code” so I think it’s useful to have done this work.

The vendor working with me on this feature has been the awesome AIAIAI who make some very nice modular headphones. A few minutes ago we uploaded the H05 v1.5 firmware to the LVFS testing stream and v1.6 will be coming soon with even more bug fixes. To update the AIAIAI H05 firmware you just need to connect the USB cable and press and hold the top and bottom buttons on the headband until the LED goes out. You can then update the firmware using fwupdmgr update or just using GNOME Software. The big caveat is that you have to be running fwupd >= 1.0.3 which isn’t scheduled to be released until after Christmas.

I’ve contacted some more vendors I suspect are using the CSR chips. These include:

  • Jarre Technologies
  • RIVA Audio
  • Avantree
  • Zebra
  • Fugoo
  • Bowers&Wilkins
  • Plantronics
  • BeoPlay
  • JBL

If you know of any other “wireless speaker” companies that have issued at least one firmware update to users, please let me know in a comment here or in an email. I will follow up all suggestions and put the status on the Naughty&Nice vendorlist so please check that before suggesting a company. It would also be really useful to know the contact details (e.g. the web-form URL, or the email address) and also the model name of the device that might be updatable, although I’m happy to google myself if required. Thanks as always to Red Hat for allowing me to work on this stuff.

Interview with Rytelier

Could you tell us something about yourself?

I’m Rytelier, a digital artist. I’ve had an interest in creating art for a few years, I mainly want to visualize my original world.

Do you paint professionally, as a hobby artist, or both?

Currently I do only personal work, but I will look for some freelance job in the future.

What genre(s) do you work in?

I work mainly in science fiction – I’m creating an original world. I like to try various things, from creatures to landscapes and architecture. There are so many things to design in this world.

Whose work inspires you most — who are your role models as an artist?

It’s hard to point out certain artists, there are so many. Mainly I get inspired by fantasy art from the internet, I explore various websites to find interesting art.

I recommend looking at my favourites gallery, there are many works that inspire me.

How and when did you get to try digital painting for the first time?

It was years ago, I’ve got interested in the subject after I saw other people’s work. It was obviously confusing, how to place strokes, how to mix colors, and I had to get used to not looking at my hand when doing something on the tablet.

What makes you choose digital over traditional painting?

I like the freedom and flexibility that digital art gives. I can create a variety of textures, find colors more easily and fix mistakes.

How did you find out about Krita?

I saw a news item about Krita on some website related to digital art and decided to try it.

What was your first impression?

I liked how many interesting brushes there were. As time went on I discovered more useful features. It was surprising to find out that some functions aren’t available in Photoshop.

What do you love about Krita?

It has many useful functions and very high user convenience. I love the brush editor – it’s clean and simple to understand, but powerful. The dynamics curve adjustment is useful, the size dependent brush with sunken curve allows me to paint fur and grass more easily.

Also different functional brush engines. Color smudge is nice for more traditional work, like mixing wet paint. Shape brush is like a lasso, but better because it shows the shape instantly, without having to use the fill tool. Filter brush is nice too, I mainly use it as sharpen and customizable burn/dodge. There are also ways to color line art quickly. For a free program that functionality is amazing — it would be amazing even for a paid program! I like this software much more than Photoshop.

What do you think needs improvement in Krita? Is there anything that really annoys you?

The performance is the thing I most want to see improved for painting and filters. I’m happy to see multi-threaded brushes in the 4.0 version. Also I would like more dynamic preview on applying filters like the gradient map, where it updates instantly when moving the color on the color wheel. It annoys me that large brush files (brushes with big textures) don’t load, I
have to optimize my textures by reducing the size so the brush can load.

What sets Krita apart from the other tools that you use?

The amount of convenience is very high compared to other programs. The amount of “this one should be designed in a better way, it annoys me” things is the smallest of all the programs I use, and if something is broken, then most of these functions are announced to improve in 4.0.

If you had to pick one favourite of all your work done in Krita so far,  what would it be, and why?

It’s hard to pick a favourite. I think this, because I challenged myself in this picture and they are my original character, which I like a lot.

What techniques and brushes did you use in it?

I use brushes that I’ve created myself from resources found on the internet and pictures scanned by myself. I like to use slightly different ways of painting in every artwork, still looking for techniques that suit me best. Generally I start from sketch, then paint splatter going all over the canvas, then adding blurry forms, then adding details. Starting from soft edges allows me to find good colors more easily.

Where can people see more of your work?

https://rytelierart.deviantart.com/
I will open galleries in other sites in the future.

Anything else you’d like to share?

I hope that Krita will get more exposure and more people, including professionals, will use it and will donate to its development team instead of buying expensive digital art programs. Open source software is having a great time, more and more tools are being created that replace these expensive ones in various categories.

December 10, 2017

Free Video Game Ideas

The ideas are free, not the games.

How about a sports (football, hockey, basketball, etc.) simulator that simulates what it’s like to play a game rather than simulating what it’s like to watch one on TV.

Or, if we’re going to simulate what feels like to watch sports on TV, let’s get hyper-real. Greasy potato-chip fingers, bathroom breaks during ads, find the remote!

December 09, 2017

Homemade Arduino Part 1: Programming an Atmega328 on a Breadboard

There are lots of tutorials around for building an Arduino on a breadboard, using an Atmega328 (or the older 168) chip, a crystal, a few capacitors and resistors and a power supply. It's a fun project that every Arduino hacker should try at least once.

But while there are lots of instructions on how to wire up a breadboard Arduino, most instructions on how to program one are confusing and incomplete.

Of course, you can program your Atmega chip while it's in an Arduino, then unplug it from the Arduino's socket and move it to the breadboard. But what a hassle! It's so more convenient to leave the chip in the breadboard while you test new versions of the code. And you can, in two different ways: with FTDI, which uses the Arduino bootloader, or with an ISP, which doesn't.

Either way, start by downloading a good pinout diagram for the Atmega328 chip. I use this one: the Arduino ATmega328 Pinout from HobbyTronics, which is very compact yet does a good job of including both the mappings to Arduino digital and analog pins and the functions like RX, TX, MOSI and MISO you'll need for programming the chip.

Load Programs with FTDI

[Circuit for Atmega328 on breadboard with FTDI friend] An FTDI board is a little trickier to wire than an ISP, but it's less risky because it loads the code the same way an Arduino would, so you don't overwrite the bootloader and you can still put your chip back into an Arduino if things go wrong. So let's start with FTDI.

I use an Adafruit "FTDI Friend", but there are lots of similar FTDI boards from Sparkfun and other vendors. They have six outputs, but you'll need only five of those. Referring to your Atmega pinout, wire up power, ground, TX, and RX. For some FTDI boards you may need pullup resistors on the TX and RX lines; I didn't need them.

Now you have four pins connected. Wiring the reset line is more complicated because it requires a 0.1μF capacitor. A lot of tutorials don't mention the capacitor, but it didn't work for me without one. Connect from RTS on the FTDI board, through the 0.1μF cap, to the RST line.

A 0.1μF capacitor is an electrolytic cap with a positive and a negative lead, but the few online tutorials that even mention the capacitor don't bother to say which side is whick. I connected the FTDI friend to the cap's negative lead, and the positive lead to the Atmega chip, and it worked.

You may also need a pullup on that RST/RTS line: a resistor around 10kΩ from the RST pin 1 of the atmega chip to the 5v power line. Note: the Fritzing diagram here shows pullup resistors on RST, TX and RX. You may not need any of them.

Incidentally, RST stands for "reset", while RTS stands for "Ready To Send"; they're not meant as anagrams of each other. The remaining pin on the FTDI friend, CTS, is "Clear To Send" and isn't needed for an Arduino.

Once the wiring is ready, plug in the FTDI board, check to make sure Port is set to whatever port the FTDI board registered, and try uploading a program as if you were uploading to a normal Arduino Uno. And cross your fingers. If it doesn't work, try fiddling with pullups and capacitor values.

Load Programs with an ISP

[Circuit for Atmega328 on breadboard with ISP] An In-System Programmer, or ISP, writes programs straight to the chip, bypassing (and overwriting) the Arduino bootloader. You can also use an ISP to burn a new bootloader and reprogram the fuses on your Arduino, to change parameters like the clock rate. (More on that in Part 2.)

You can use an Arduino as an ISP, but it's somewhat unreliable and prone to unexplained errors. A dedicated ISP isn't expensive, is easier to wire and is more likely to work. A common type of ISP is called a "USBtinyISP", and you can buy one from vendors like Sparkfun or Adafruit, or search for usbtinyisp on sites like ebay or aliexpress.

Update: I've been curious about this flakiness: why does "Arduino as ISP" work fine for some people and utterly fail for others? One person I asked thought it had to do with the way Arduinos reset the RESET line whenever the serial port is opened: so RESET gets toggled at the wrong time as the bootloader code is being transferred. An alternate method that may get around this is Gammon Forum's Atmega bootloader programmer, which includes the bootloader bits as part of the code so it doesn't need to re-open the serial port. Someone else says a 10 uF capacitor between reset and ground should prevent that from happening; and another person says it should be a 100nF capacitor between RST on the programmer and RST on the AVR-chip plus a 10k pullup resistor, Most Arduino-as-ISP tutorials, including the official ones on arduino.cc, don't mention either capacitors or pullups, so that may explain why the method works for some people and not others.

Arduino ISP pinout ISPs typically use a six-pin connector (2x3). It's not always easy to figure out which end is which, so use a multimeter in continuity mode to figure out which pin is ground. Once you're sure, mark your connector so you'll know which pin is pin 1 (MISO, the pin opposite ground).

Once you have your ISP pins straight, refer to your handy-dandy Atmega328 pinout and connect power, ground, MOSI, MISO, SCK, and RST to the appropriate Atmega pins.

All wired up? In the Arduino IDE, set Programmer to your ISP, for instance, USBtinyISP or Arduino as ISP Then use the Upload button to upload sketches. If you prefer Arduino-mk instead of the IDE, add this to your Makefile:

ISP_PROG     = usbtiny
(or whatever ISP you're using). Then type make ispload instead of make upload

Once you have your FTDI or ISP working, then you can think about making an even simpler circuit -- without the external clock and its associated capacitors. But there are a couple of additional tricks to that. Stay tuned for Part 2.

December 07, 2017

OARS Gets a New Home

The Open Age Ratings Service is a simple website that lets you generate some content rating XML for your upstream AppData file.

In the last few months it’s gone from being hardly used to being used multiple times an hour, probably due to the requirement that applications on Flathub need it as part of the review process. After some complaints, I’ve added a ton more explanation to each question and made it easier to use. In particular if you specify that you’re creating metadata for a “non-game” then 80% of the questions get hidden from view.

As part of the relaunch, we now have a proper issue tracker and we’re already pushed out some minor (API compatible) enhancements which will become OARS v1.1. These include several cultural sensitivity questions such as:

  • Homosexuality
  • Prostitution
  • Adultery
  • Desecration
  • Slavery
  • Violence towards places of worship

The cultural sensitivity questions are work in progress. If you have any other ideas, or comments, please let me know. Also, before I get internetted-to-death, this is just for advisory purposes, not for filtering. Thanks.

December 06, 2017

UTC and Anywhere on Earth support

A quick post to tell you that we finally added UTC support to Clocks' and the Shell's World Clocks section. And if you're into it, there's also Anywhere on Earth support.

You will need to have git master versions of libgweather (our cities and timezones database), and gnome-clocks. This feature will land in GNOME 3.28.



Many thanks to Giovanni for coming up with an API he was happy with after I attempted a couple of iterations on one. Enjoy!

Update: As expected, a bug crept in. Thanks to Colin Guthrie for spotting the error in the "Anywhere on Earth" timezone. See this section for the fun we have to deal with.

December 05, 2017

Simple Exposure Mapping in GIMP


Simple Exposure Mapping in GIMP

A simple approach to blending exposures

There are many different approaches to blending exposures in the various projects, and they can range from extremely detailed and complex to quick and simple. Today we’re going to look at the latter.

I was recently lucky enough to attend an old friends wedding in upstate NY. Mairi got married! (For those not familiar with her, she’s the model from An Open Source Portrait as well as A Chiaroscuro Portrait tutorials.)

Mairi Chiaroscuro Portrait Mairi’s chiaroscuro portrait.

I had originally planned on celebrating with everyone and wrangling my two kids, so I left my camera gear at home. Turns out Mairi was hoping that I’d be shooting photos. Not wanting to disappoint, I quickly secured a kit from a local rental shop. (Thank goodness for friends new and old to help wrangle a very busy 2 year old.)

During the rehearsal I was experimenting with views to get a feel for the room I’d have and how framing would work out. One of the shots looked from the audience and right into a late afternoon sun. My inner nerd kicked in and I thought, “This might be a neat image to use for a tutorial!”.

Exposure Fusion (Mapping)

The idea behind exposure fusion (or mapping) is to extend the dynamic range represented in an image by utilizing more than one exposure of the same subject and choosing relevant bits to fit in the final output.

I say “fit” because usually you are trying to put a larger amount of data than a single image might have been able to capture. So you choose which parts you want to use to get something that you’ll like. For example, here we have two images that will be used in this article where the image showing the foreground correctly causes the sky to blow out, while exposing for the sky causes the foreground to go almost black. By selectively combining these two images, we can get something that might show a larger dynamic range than would have been possible in a single exposure:

Sample of exposure fusion results This porridge is too hot, this porridge is too cold, but this porridge is just right.

This is one common use case scenario for creating HDR/EXR imaging (and tonemapping is the term for doing exactly what we’re describing here - squishing data into the viewable range in a way that we like). In fact, at the end of this article I’ll show how Enfuse handled merging these image exposures (spoiler: pretty darn well).

Exposing

In exposing for the subjects in this image, I used the structure to block the sun from direct view (though there’s still a loss of contrast and flaring). The straight out of the camera jpg looks like this:

Foreground Exposure Foreground exposure

This gave me a well exposed foreground and subjects. I then gave the shutter speed a quick spin to 11000 (about 4-stops) to get the sky better exposed. The camera jpg for the sky looked like this:

Sky Exposure Sky exposure

In retrospect I probably would have been better to shoot for 2-stops difference to keep the sky exposed higher in the histogram, but c’est la vie. It also helps to avoid going too far in the extremes when exposing, so as to avoid making it look too unrealistic (yes - the example above probably skirts that pretty close, but it’s exaggerated to make a good article). This gives us a nice enough starting point to play with some simple exposure mapping.

Alignment

For this to work properly the images do need to line up as perfectly as possible. Imperfect alignment can be worked around to some extent, usually determined by how complex your masking will have to be, but the better aligned the images are the easier your job will be.

As usual I have had good luck using the align_image_stack script that’s included as part of Hugin. This makes short work of getting the images aligned properly for just this sort of work:

/path/to/hugin/align_image_stack -m -a OUT FILE1 FILE2

On Windows, this looks like:

c:\Program Files\Hugin\bin\align_image_stack.exe -m -a OUT FILE1 FILE2

Once it finishes up, you’ll end up with some new files like OUT0001.tif. These are your aligned images that we’ll now bring into GIMP!

Masking

The heart of fusing these exposures is going to rely entirely on masking. This is where it usually pays off nicely to take your time and consider carefully an approach that will keep things looking clean and with a natural transition between the exposures.

If this had simple geometry it would be an easy problem to solve, but the inclusion of the trees in the background makes it slightly more complex (but not nearly as bad as hair masking can get). The foreground and sky are very simple things to mask overall, where we know we may want 100% of the foreground to come from one image, and 100% of the sky from another. This helps simplifies things greatly.

Rough Mask Example Rough mask (temporary - for reference only) where we can see we want all of the sky from one image, and all of the foreground from another.

I tend to keep the darker sky layer on the bottom and the lighter foreground layer above that.

The hard edges of the structure make it an easy masking job there, so the main area of concern here is getting a good blend with the treeline in the background. There are a couple of approaches we can take to try and get a good blend, so let’s have a look…

Luminosity (Grayscale) Mask

A common approach would be to apply an inverted grayscale mask to the foreground layer. If there’s a decent amount of contrast between the foreground/sky layers then this is a quick and easy way to get something to use as a base for further work:

GIMP Add Layer Mask Dialog

Applying this mask yields pretty good looking results right away:

GIMP Inverted Grayscale Mask

You can also investigate some of the other color channel options to see if there might be something that works better to create a clean mask with. In GIMP 2.9.x I also found that using Colors > Components > Extract Component using CMYK Key produced another finely separated option.

This makes a nice starting point. As we said we wanted all of the sky from one exposure and the rest of the image from the other exposure, we can start roughing in the overall mask.

For this simple walkthrough we can make our job a bit easier by using two copies of the foreground layer and selectively blending them over the sky layer.

Why Two Layers?

If you take your foreground layer and start cleaning up the mask for the sky by painting in black, it should be relatively easy. Until you get down to the treeline. You can use a soft-edged brush and try to blend in smoothly, but in order to let the sky come through nicely you may find yourself getting more of the dark exposure trees showing through. This will show as a dark halo on the tops of the trees:

GIMP Mask Dark Halo

A nice way to adjust the falloff along the tops of the trees is by using a second copy of the foreground layer, and using a gradient on the mask that will blend smoothly from full sky to full foreground along the tops of the trees. This will ease the dark halo a bit until the transition looks good to you. You can then modify/update the gradient on the copy until the transition is smooth or to your liking.

At this point my layers would look like this in GIMP:

GIMP Layer Stack Blending

The results of using a second fore layer with a gradient to help ease the transition:

GIMP Mask Original GIMP Mask Dark Halo GIMP Mask Gradient Top: Original grayscale mask only
Middle: Manual mask painting down to treeline
Bottom: Second layer with gradient mask

When pixel-peeping it may not seem perfect, but in the context of the entire image it’s a nice way to get an easy blend for not too much extra work.

GIMP Mask Grayscale vs Gradient Left: Grayscale mask, Right: final mask with gradient

At this point most of the exposure is blended nicely. The only place in this particular image I would work some more would be to darken the structure a little bit to lessen the flaring and maybe bring back a little darkness.

This can be accomplished by painting over the top mask. I used black with a smaller opacity of around 25% to paint over the structure and allow the darker version underneath to show through a bit more:

GIMP Mask Darkened on Structure Left: gradient masked, Right: structure darkened slightly to taste

Enfuse Comparison

I have previously used Enfuse and gotten great results even more quickly. For comparison here is the result of running Enfuse against the same images:

Enfuse compared to our manual masking Left: Manual masking result, Right: Enfuse automatic blend

I prefer our manually blended result personally, but I could see another future article or post about using the Enfuse blend for areas of complexity and blending the Enfuse output into the final image to help. Might be interesting.

(I prefer our blended result because Enfuse considered extremely bright areas as candidates for fusion with the other exposure - so the bricks and tent highlights got pushed down automatically.)

Fin

From here any further fiddling with the image is purely for fun. The two versions of the image have been merged nicely. If you wanted to adjust the result to not appear quite as extreme you could modify each of the layers to taste.

For instance, you could lighten the sky layer to decrease the extreme range difference between it and the foreground layer. Where’s the fun in keeping it too realistic though? :)

For reference, here’s my final version after masking with a bit of a Portra tone thrown in for good measure:

Final version with portra color toning Not bad for a relatively quick approach.

Resources

This wouldn’t be complete with some resources and further reading for folks!

I have saved the GIMP 2.9.x .XCF file I used to write this. It has all of the masks and layers I used to create the final version of the image:

Further Reading

@McCap also created a YouTube video walking through how he approached exposure blending:

The image Wedding Rehearsal by Pat David is licensed under a Creative Commons Attribution-ShareAlike 4.0 International License.
Permissions beyond the scope of this license may be available at https://patdavid.net/about.

December 04, 2017

Distrinet R&D Bites

The Distrinet Research Group at KULeuven (where I studied!), recently asked me to speak about “Cloud Native” at one of their R&D Bites sessions. My talk covered Kubernetes, cloud automation and all the cool new things we can do in this brave new cloud native world.

Annotated slides of the talk can be found here.

Experiences in building cloud-native businesses: the Ticketmatic case


Comments | More on rocketeer.be | @rubenv on Twitter

Los Alamos Raspberry Pi Club Starting Up Thursday

[Raspberry Pi Zero W with LED] Are you interested in all things Raspberry Pi, or just curious about them? Come join like-minded people this Thursday at 7pm for the inaugural meeting of the Los Alamos Raspberry Pi club!

At Los Alamos Makers, we've had the Coder Dojo for Teens going on for over a year now, but there haven't been any comparable programs that welcomes adults. Pi club is open to all ages.

The format will be similar to Coder Dojo: no lectures or formal presentations, just a bunch of people with similar interests. Bring a project you're working on, see what other people are working on, ask questions, answer questions, trade ideas and share knowledge.

Bring your own Pi if you like, or try out one of the Pi 3 workstations Los Alamos Makers has set up. (If you use one of the workstations there, I recommend bringing a USB stick so you can save your work to take home.)

Although the group is officially for Raspberry Pi hacking, I'm sure many attendees will interested in Arduino or other microcontrollers, or Beaglebones or other tiny Linux computers; conversation and projects along those lines will be welcome.

Beginners are welcome too. You don't have to own a Pi, know a resistor from a capacitor, or know anything about programming. I've been asked a few times about where an adult can learn to program. The Raspberry Pi was originally introduced as a fun way to teach schoolchildren to program computers, and it includes programming resources suitable to all ages and abilities. If you want to learn programming on your own laptop rather than a Raspberry Pi, we won't turn you away.

Raspberry Pi Club: Thursdays, 7pm, at Los Alamos Makers, 3540 Orange Street (the old PEEC location), Suite LV1 (the farthest door from the parking lot -- look for the "elevated walkway" painted outside the door).

There's a Facebook event: Raspberry Pi club on Facebook. We have meetings scheduled for the next few Thursdays: December 7, 14, and 21, and after that we'll decide based on interest.

ColorHug Plus Update

Here’s an update for people waiting for news on the ColorHug+ spectrophotometer, and perhaps not the update that you were hoping for. Three things have recently happened, and each of them makes producing the ColorHug+ even harder than it was before:

  1. A few weeks I became a father again. Producing the ColorHug and ColorHug2 devices takes a significant amount of time, brain, muscle and love, and I’m still struggling with dividing up my time between being a modern hands-on dad and also a full time job at Red Hat. ColorHug was (and still is) a hobby that got a little out of control, and not something that brings in any significant amount of money. A person spending £300 on a complex device is going to expect at least some level of support, even when I’ve had no sleep and only have half a brain on a Saturday morning.
  2. Brexit has made the GBP currency plunge in value over the last 12 months, which in theory should be good as it will encourage exports. What’s slightly different for me is that 80% of the components for each device are purchased in USD and EUR, and the remaining ones in GBP have risen accordingly with the currency plunge. I have no idea what a post-Brexit Britain looks like, but I think it’s a prudent choice to not “risk” £20k in an investment I’d essentially hope to break even on long term, for fun.
  3. The sensor for the ColorHug+ was going to be based on the bare chip SPARK from OceanOptics. I’ve spent a long time working out all the quirks of the sensor, making it work with a UV and wideband illuminant and working out all the packaging questions. The price of the sensor was always going to be expensive (it was greater than half of the RRP in one component alone, even buying a massive batch) but last month I got an email saying the sensor was going to be discontinued and would no longer be available. This is figuratively and also quite literally back to the drawing board.

I’ve included some photos above to show I’ve not been full of hot air for the last year or so, and to remind everyone that the PCB, 3D light guide model and client software are all in the various ColorHug git repos if you want to have a go at building one yourself (although, buy the sensor quickly…). I’ll still continue selling ColorHug2 devices, and supporting all the existing hardware but this might be the end of the line for ColorHug spectrometer. I’ll keep my eye on all the trade magazines for any new sensor that is inexpensive, reliable and accurate enough for ICC profiles, so all this might just be resurrected in the future, but for the short term this is all on ice. If you want a device right now the X-Rite i1Studio is probably the best of the bunch, although it is sold by Pantone with an RRP of £450. Fair warning: Pantone and free software are not exactly bedfellows, although it does work with ArgyllCMS using a reverse engineered userspace driver that might void your warranty.

I’ll update the website at some point this evening, I’m not sure whether to just post all this or remove the ColorHug+ page completely. Perhaps a sad announcement, but perhaps not one that’s too unexpected considering the lack of updates in the last few months. Sorry to disappoint everybody.

December 03, 2017

darktable 2.4.0rc0 released

we’re proud to announce the first release candidate for the upcoming 2.4 series of darktable, 2.4.0rc0!

the github release is here: https://github.com/darktable-org/darktable/releases/tag/release-2.4.0rc0.

as always, please don’t use the autogenerated tarball provided by github, but only our tar.xz. the checksum is:

$ sha256sum darktable-2.4.0rc0.tar.xz
66795f96dfd46b921a006836eb062f40cab1e93d018f61ccb7e650fb01a0016d darktable-2.4.0rc0.tar.xz
$ sha256sum darktable-2.4.0rc0.dmg
23894c0ec808c8420719646ee289aba68fc15761ce812358ba3456691ad5849c darktable-2.4.0rc0.dmg
$ sha256sum darktable-2.4.0rc0.exe
ed560de786340cbdd94e446615cec8eef52fbbeb3ac81f7d10edfeee1e5b74ee darktable-2.4.0rc0.exe

Important note: to make sure that darktable can keep on supporting the raw file format for your camera, please read this post on how/what raw samples you can contribute to ensure that we have the full raw sample set for your camera under CC0 license!

and the changelog as compared to 2.2.0 can be found below. Some of the fixes might have been backported to the stable 2.2.x series already.

  • The maintainership of the RawSpeed library was transferred to the darktable project. The work on code cleanup, hardening, modernization, simplification and testing is ongoing.
  • Well over 2 thousand commits to darktable+rawspeed since 2.2.0
  • 244 pull requests handled
  • 320+ issues closed
  • Updated user manual is coming soon™

Hell Froze Over

  • As you might have read on our news post we finally ported darktable to Windows and intend to support it in the future. At the moment it’s still lacking a few features (for example there is not printing support), has a few limitations (tethering requires special drivers to be installed) and comes with its own set of bugs. But overall we are confident that it’s quite usable already and hope you will enjoy it. A very special thanks goes to Peter Budai who finally convinced us to agree to the port and who did most of the work.

The Big Ones

  • A new module for haze removal
  • The local contrast module can now be pushed much further, it also got a new local laplacian mode
  • Add undo support for masks and more intelligent grouping of undo steps
  • Blending now allows to display individual channels using false colors
  • darktable now supports loading Fujifilm compressed RAFs
  • darktable now supports loading floating point HDR DNGs as written by HDRMERGE
  • We also added channel specific blend modes for Lab and RGB color spaces
  • The base curve module allows for more control of the exposure fusion feature using the newly added bias slider
  • The tonecurve module now supports auto colour adjustment in RGB
  • Add absolute color input as an option to the color look up table module
  • A new X-Trans demosaicing algorithm, Frequency Domain Chroma, was implemented.
  • You can now choose from pre-defined scheduling profiles for OpenCL
  • Speaking of OpenCL, darktable now allows to force-use OpenCL for a specific pixelpipe
  • Xmp sidecar files are no longer written to disk when the content didn’t actually change. That mostly helps with network storage and backup systems that use files’ time stamps

New Features And Changes

  • Show a dialog window that tells when locking the database/library failed
  • Don’t shade the whole region on the map when searching for a location. Instead just draw a border around it.
  • Also in map mode: Clear the search list and map indicators when resetting the search module.
  • With OsmGPSMap newer than version 1.1.0 (i.e., anything released after that OsmGPSMap version) the map will show copyright info.
  • Running jobs with a progressbar (mostly import and export) will show that progress bar ontop the window entry in your task bar – if the system supports it. It should work on GNOME, KDE and Windows at least.
  • Add bash like string replacement for variables (export, watermark, session settings).
  • Add a preferences option to ask before removing empty dirs
  • The “colorbalance” module got a lot faster, thanks to SSE optimized code
  • Make gradient sliders a little more colorful and use them in the white balance module
  • Make PNG compression level used for exporting configurable
  • On OSX, load single images from command line or via drag&drop in darkroom mode
  • Add an option to omit the intermediate tag hierarchy in exported files and only add the last level
  • In the watermark module, sort the list of SVG files and omit the file extension
  • Support XYZ as a proofing profile
  • Local contrast now got a new slider to set the midtone range
  • darktable got two new helper scripts (those are not installed by default, grab them from the sources): One to purge thumbnails that no longer have an associated image in the database, and a second script that uses inotify to watch a folder for new files to open them in a running darktable instance.
  • In the curve editors of base curve and tone curve you can now delete nodes with a right click and see coordinates of nodes while editing. Note that you can use keyboard modifiers ctrl and shift to change the precision of your changes
  • Creating a new instance of a module can now be done with a quick click of the middle mouse button on the multi-instance icon
  • New darktable installations on computers with more than 8 Gb of memory will now by default use half of that per module
  • Several background colors and the brush color are now configurable in the CSS
  • Some new cameras can bump the ISO level to insane highs. We try to follow as good as we can by no longer limiting it to 51200 in the GUI
  • Base curve and the highlights module now support multiple instances and use blending and masks
  • Having the 1 key toggle between 1 and 0 stars wasn’t very popular with many people. You can disable that extra feature and have it behave like the other rating shortcuts now
  • You can decide if you want to be asked before resetting the history stacks of images from the lighttable
  • The grain module was slightly changed to have a more pleasing, photographic-paper like appearance
  • Using the color look up table module you can now convert your images to monochrome, honoring the Helmholtz-Kohlrausch effect
  • Some more small improvements were made

Bugfixes

  • Fix the problem with rating images by accident when moving the mouse while typing an image size in the export module
  • Fix several oddities in folder and tag mode of the collect module.
  • Print mode’s color profile settings no longer interact with the export module
  • Update the style lists when importing a style
  • Fix some bugs with multiple module instances used in a style
  • On OSX only the main window should be fullscreen, not the popups
  • Some speedups with VERY big libraries or having A LOT OF tags
  • Significantly speed up tagging many images
  • Fix searching locations using OpenStreetMap
  • Fix partial copies of large files in “import from camera”
  • Fix a crash in the import dialog when using Lua to add widgets there
  • Fix some false-positive warnings about another running darktable instance and it having locked the databases
  • No longer switch to the favourite modules group when duplicating one of its modules
  • Fix loading of XYZ files
  • Fix Lab export when the profile was set from the lighttable
  • Create tmp snapshot files with mode 0600 to stop other people looking at them
  • Fix several bugs with Wayland. However, there are still issues, so darktable will prefer XWayland
  • Google deprecated the Picasa Web API so it’s no longer possible to create G+ albums
  • Fix the default for sliders with target not being “red” in the channel mixer
  • Fix the removing of directories
  • Make the escape key cancel history dialogs
  • Block keyboard accels when editing camera controls
  • Properly delete XMP sidecars
  • Make sure that the rating set in darktable is used for the exported file, not something set inside the raw file
  • Don’t re-write all XMP files when detaching a tag
  • Sync XMPs when a tag is removed from the database
  • Sync XMPs after a tag is attached/detached via the Lua API
  • Many more bugs got fixed

Lua

  • darktable now uses Lua 5.3. The bundled copy got updated accordingly
  • Add dt.print_log. It’s like print_error but without the ERROR prefix
  • Reorder callback parameters for intermediate export image: add the actual image to the parameters of the event
  • Call lua post-import-image event synchronously
  • Add darktable.configuration.running_os to detect the OS darktable is running on
  • New widget type: section_label, adds a label which looks like a section change

Changed Dependencies

  • CMake 3.1 is now required.
  • In order to compile darktable you now need at least gcc-4.9+/clang-3.4+, and gcc-5.0+ is highly recommended.
  • ZLIB is now required for the DNG Deflate compressed raw support.
  • darktable now uses Lua 5.3

Camera support, compared to 2.2.0

Warning: support for Nikon NEF ‘lossy after split’ raws was unintentionally broken due to the lack of such samples. Please see this post for more details. If you have affected raws, please contribute samples!

Base Support

  • Canon EOS 200D
  • Canon EOS Kiss X9
  • Canon EOS Rebel SL2
  • Canon EOS 6D Mark II (sRaw1, sRaw2)
  • Canon EOS 77D
  • Canon EOS 9000D
  • Canon EOS 800D
  • Canon EOS Kiss X9i
  • Canon EOS Rebel T7i
  • Canon EOS M5
  • Canon EOS M6
  • Canon PowerShot G9 X Mark II
  • Canon PowerShot SX40 HS (dng)
  • Fujifilm GFX 50S (compressed)
  • Fujifilm X-A3
  • Fujifilm X-E2S
  • Fujifilm X-E3 (compressed)
  • Fujifilm X-Pro2 (compressed)
  • Fujifilm X-T2 (compressed)
  • Fujifilm X-T20 (compressed)
  • Fujifilm X100F (compressed)
  • GITUP GIT2P (chdk-a, chdk-b)
  • Kodak EasyShare Z980
  • LG D855 (dng)
  • LG H815 (dng)
  • LG Nexus 5X (dng)
  • LG US996 (dng)
  • LG VS995 (dng)
  • Leica D-LUX (Typ 109) (4:3, 3:2, 16:9, 1:1)
  • Leica X2 (dng)
  • Nikon COOLPIX B700 (12bit-uncompressed)
  • Nikon D500 (14bit-uncompressed, 12bit-uncompressed)
  • Nikon D5600 (12bit-compressed, 12bit-uncompressed, 14bit-compressed, 14bit-uncompressed)
  • Nikon D7500 (12bit-compressed, 14bit-compressed)
  • Nikon D850 (12bit-compressed, 12bit-uncompressed, 14bit-compressed, 14bit-uncompressed)
  • Nikon LS-5000 (dng)
  • Nokia Lumia 1020 (dng)
  • Olympus E-M10 Mark III
  • Olympus E-M1MarkII
  • Olympus TG-5
  • Panasonic DC-FZ82 (4:3)
  • Panasonic DMC-FZ80 (4:3)
  • Panasonic DMC-FZ85 (4:3)
  • Panasonic DC-GH5 (4:3)
  • Panasonic DC-FZ91 (4:3)
  • Panasonic DC-FZ92 (4:3)
  • Panasonic DC-FZ93 (4:3)
  • Panasonic DC-TZ90 (4:3)
  • Panasonic DC-ZS70 (4:3)
  • Panasonic DMC-FZ330 (4:3)
  • Panasonic DMC-GF6 (16:9, 3:2, 1:1)
  • Panasonic DMC-TZ61 (4:3, 3:2, 1:1, 16:9)
  • Panasonic DMC-ZS40 (4:3, 3:2, 1:1, 16:9)
  • Panasonic DMC-TZ80 (4:3)
  • Panasonic DMC-TZ81 (4:3)
  • Panasonic DMC-TZ85 (4:3)
  • Panasonic DMC-ZS60 (4:3)
  • Pentax K-5 (dng)
  • Pentax K-r (dng)
  • Pentax K10D (dng)
  • Samsung G920F
  • Samsung G935F
  • Samsung GX10
  • Sony ILCE-6500
  • Sony ILCE-9

White Balance Presets

  • Canon EOS 6D Mark II
  • Fujifilm X-T20
  • Fujifilm X100F
  • Nikon 1 AW1
  • Nikon Coolpix A
  • Panasonic DMC-GX80
  • Panasonic DMC-GX85
  • Panasonic DMC-TZ100
  • Panasonic DMC-TZ101
  • Panasonic DMC-TZ110
  • Panasonic DMC-ZS110
  • Pentax K-3 II

Noise Profiles

  • Canon EOS 1300D
  • Canon EOS Kiss X80
  • Canon EOS Rebel T6
  • Canon EOS 5D Mark IV
  • Canon EOS 6D Mark II
  • Canon EOS M5
  • Canon PowerShot G16
  • Canon PowerShot G3 X
  • Canon PowerShot G7 X Mark II
  • Canon PowerShot G9 X Mark II
  • Fujifilm X-M1
  • Fujifilm X-Pro1
  • Fujifilm X-T20
  • Leica X2
  • Nikon Coolpix A
  • Nikon D2X
  • Nikon D3000
  • Nikon D3400
  • Nikon D500
  • Olympus E-M1MarkII
  • Olympus E-P5
  • Panasonic DMC-FZ200
  • Panasonic DMC-FZ300
  • Panasonic DMC-G7
  • Panasonic DMC-G70
  • Panasonic DMC-G8
  • Panasonic DMC-G80
  • Panasonic DMC-G81
  • Panasonic DMC-G85
  • Panasonic DMC-GX80
  • Panasonic DMC-GX85
  • Panasonic DMC-LX100
  • Panasonic DMC-TZ100
  • Panasonic DMC-TZ101
  • Panasonic DMC-TZ110
  • Panasonic DMC-ZS110
  • Pentax K-70
  • Sony DSC-RX100M5
  • Sony ILCA-68
  • Sony ILCE-5000
  • Sony ILCE-6500

Updated Translations

  • Dutch
  • French
  • German
  • Hebrew
  • Russian
  • Spanish

November 30, 2017

Programming an ATtiny85, Part 2: With the Arduino Software (and a Makefile)

Having written a basic blink program in C for my ATtiny85 with a USBtinyISP (Part 1), I wanted to use it to control other types of hardware. That meant I wanted to be able to use Arduino libraries.

The Arduino IDE

I normally use Makefiles, but the Arduino IDE is much better supported so I tried that first. I followed the steps at High-Low Tech: Programming an ATtiny w/ Arduino 1.6 (or 1.0). But the short summary is:

  • Run the Arduino IDE
  • File->Preferences
  • In "Additional Boards Manager" near the bottom, paste this:
    https://raw.githubusercontent.com/damellis/attiny/ide-1.6.x-boards-manager/package_damellis_attiny_index.json
    and click OK
  • Tools->Boards->Board Manager...
  • Find the ATTiny entry, click on it, and click Install
  • Back in the main Arduino IDE, Tools->Boards should now havea couple of Attiny entries. Choose the one that corresponds to your ATTiny; then, under Processor, narrow it down further.

In Tools->Programmer, choose the programmer you're using (for example, USBtinyISP).

Now you should be able to Verify and Upload a blink sketch just like you would to a regular Arduino, subject to the pin limitations of the ATTiny.

That worked for blink. But it didn't work when I started adding libraries. Since the command-line was what I really cared about, I moved on rather than worrying about libraries just yet.

ATtiny with Arduino-Makefile

For most of my Arduino development I use an excellent package called Arduino-Makefile. There's a Debian package called arduino-mk that works fine for normal Arduinos, but for ATtiny, there have been changes, so use the version from git. A minimal blink Makefile looks like this:

BOARD_TAG = uno
include /usr/share/arduino/Arduino.mk

It assumes that if you're in a directory called blink, it should compile a file called blink.ino. It will also build any additional .cpp files it finds there. make upload uploads the code to a normal Arduino.

With Attiny it gets quite a bit more complicated. The key is that you have to specify an alternate core:

ALTERNATE_CORE = ATTinyCore

But there are lots of different ATtiny cores, they're all different, and they each need a different set of specifiers like BOARD_TAG in the Makefile. Arduino-Makefile comes with an example, but it isn't very useful since it doesn't say where to get the cores that correspond with the various samples. I ended up filing a documentation bug and exchanging some back-and-forth with the maintainer of the package, Simon John, and here's what I learned.

First: as I mentioned earlier, you should use the latest git version of Arduino-Makefile. The version in Debian is a little older and some things have changed; while the older version can be made to work with ATtiny, the recipes will be different from the ones here.

Second, the recipes for each core will be different depending on which version of the Arduino software you're using. Simon says he sticks to version 1.0.5 when he uses ATtinys, because newer versions don't work as well. That may be smart (certainly he has a lot more experience than I do), but I'm always hesitant to rely on software that old, so I wanted to get things working with the latest Arduino, 1.8.5, if i could, so that's what the recipes here will reflect.

Third, as mentioned in Part 1, clock rate should be 1MHz, not 8MHz as you'll see in a lot of web examples, so: F_CPU = 1000000L

Fourth, uploading sketches. As mentioned in the last article, I'm using a USBtinyISP. For that, I use ISP_PROG = usbtiny and sketches are uploaded by typing make ispload rather than the usual make upload. change that if you're usinga different programmer.

With those preliminaries over: I ended up getting two different cores working, and there were two that didn't work. Install the cores in subdirectories in your ~/sketchbook/hardware directory. You can have multiple cores installed at once if you want to test different cores. Here are the recipes.

CodingBadly's arduino-tiny

This is the core that Simon says he prefers, so it's the one I'm going to use as my default. It's at https://github.com/Coding-Badly/arduino-tiny.git, and also a version on Google Code. (Neither one has been updated since 2013.)

git clone it into your sketchbook/hardware. Then either cp 'Prospective Boards.txt' boards.txt or create a new boards.txt and copy from 'Prospective Boards.txt' all the boards you're interested in (for instance, all the attiny85 definitions if attiny85 is the only attiny board you have).

Then your Makefile should look something like this:

ARDUINO_DIR = /path/to/arduino-1.8.5

BOARD_TAG = attiny85at8
ALTERNATE_CORE = tiny
F_CPU = 1000000L
ISP_PROG = usbtiny

include /path/to/Arduino-Makefile/Arduino.mk

If your Arduino software is installed in /usr/share/arduino you can omit the first line.

Now copy blink.ino -- of course, you'll have to change pin 13 to be something between 1 and 6 since that's how many pins an ATtiny has -- and try make and make ispload.

SpenceKonde's ATTinyCore

This core is at https://github.com/SpenceKonde/ATTinyCore.git. I didn't need to copy boards.txt or make any other changes, just clone it under sketches/hardware and then use this Makefile:

ARDUINO_DIR = /path/to/arduino-1.8.5

ALTERNATE_CORE = ATTinyCore
BOARD_TAG = attinyx5
BOARD_SUB = 85
F_CPU = 1000000L
ISP_PROG = usbtiny

include /path/to/Arduino-Makefile/Arduino.mk

Non-working Cores

There are plenty of other ATtiny cores around. Here are two that apparently worked once, but I couldn't get them working with the current version of the tools. I'll omit links to them to try to reduce the probability of search engines linking to them rather than to the more up-to-date cores.

Damellis's attiny (you may see this referred to as HLT after the domain name, "Highlowtech"), on GitHub as damellis/attiny, was the first core I got working with Debian's older version of arduino-mk and Arduino 1.8.4. But when I upgraded to the latest Arduino-Makefile and Arduino 1.8.5, it no longer worked. Ironic since an older version of it was the one used in most of the tutorials I found for using ATtiny with the Arduino IDE. Simon says this core is buggy: in particular, there are problems with software PWM.

I also tried rexxar-tc's arduino-tiny.core2 (also on GitHub). I couldn't get it to work with any of the Makefile or Arduino versions I tried, though it may have worked with Arduino 1.0.

With two working cores, I can get an LED to blink. But libraries are the point of using the Arduino framework ... and as I tried to move beyond blink.ino, I found that not all Arduino libraries work with ATtiny. In particular, Wire, used for protocols like I2C to talk to all kinds of useful chips, doesn't work without substantial revisions. But that's a whole separate topic. Stay tuned.

November 29, 2017

An OpenHardware 1-port Hub?

I’ve spent the last couple of evenings designing an OpenHardware USB 2.0 1-port hub tentatively called the ColorHub (although, better ideas certainly welcome). Back a bit: What’s the point in a 1-port hub?

The finished device is a Cypress 2 port hub internally, with a PIC microcontroller hard wired onto the other “fixed” port. The microcontroller can control the hub and the USB power of the other “removable” port, so you can simulate an unplug, replug or hub reset using a few simple commands. This also allows you forcefully reset hardware that’s not responding, and also allows you to add hardware tests to test enumeration and device removal. With the device it is trivial to write a script to replug a device 5000 times over one evening, or re-connect a USB device that’s not responding for whatever reason. The smart hub also reports when USB devices are connected to the downstream port, and even when they have not enumerated correctly. There could be commands to get the status of that and also optionally wait until those things have happened.

The other killer feature for me is that the microcontroller has lots of spare analog and digital IO, and with two included solid-state MOSFET relays you can wire up two physical switches so that no user interaction is required. This means you can test hardware that has these kind of requirements:

  • Remove USB plug
  • Press and hold buttons A&B
  • Insert USB plug
  • Release buttons A&B

It would be fairly trivial to wire up the microcontroller ADC to get a rough power consumption figure, or to set some custom hub descriptors; it would be completely open and “hackable” like the ColorHug.

I’ve made just one prototype and am using it quite nicely in the fwupd self tests, but talking to others yesterday this seems the kind of device that would be useful for other people doing similar QA activities. I need to build another 2 for the other devices requiring manual button-presses in the fwupd hardware cardboard-box-tests and it’s exactly the same price to order 50 tiny PCBs as 5.

The dangerous question: Would anyone else be interested in purchasing this kind of thing? The price would be in the £50-60 range, so certainly not cheap, but this is really the cost of ultra-small batches of moderately complicated electronics these days. If you’re interested, send me an email (richard_at_hughsie_dot_com) and depending on demand I’ll either design some nice custom PCBs or just hack together two more prototypes for my own use. Please also tell me if something like this already exists: if so I can save some time and just buy something that someone else has built. Comments welcome.

November 27, 2017

London UX Hackfest

London UX Hackfest from jimmac on Vimeo.

Thanks to the GNOME Foundation, a handful of designers and developers got together last week in London to refocus on the core element of the GNOME experience, the shell. Allan and Cassidy have already summed up everything in their well written blog posts, so I’d like to point to some pretty pictures and the video above.

Stay tuned for some higher fidelity proposals in the areas of app switching & launching and the lock/login experience.

Krita Development Sprint 2017

With all the turmoil the project experienced in 2017 it looked for a while as if we wouldn’t have a face to face meeting this year. But that’s not good for a project working on its fourth major release! We knew we really had to sit together, and finally managed to have a smaller than usual, but very productive, sprint in Deventer, the Netherlands from Thursday 23th to Sunday 26th.

Not having been together since August 2016, we had an agenda stuffed with a enormous backlog of items. And since we’ve been working on new code for a long time ago, our bug tracker was also slowly dying from elephantiasis of the database.

Let’s do the bug tracker first: we managed to close over 120 bugs! Not every bug that gets closed gets closed with a fix: the problem is that most bug reports are actually help requests from users, and many of the rest are duplicates, or requests for features that are irrelevant for Krita. Still, while triaging the list of open and unconfirmed bug reports, we managed to fix more than a dozen real bugs.

Here’s the proof:

Now let’s go to the other topics that were discussed

Krita 4.0

We finalized the set of features that we want to complete for 4.0, and came up with a schedule. Originally, we wanted to release 4.0 this year, but that’s completely impossible…

We still want to add:

  • Lazy brush for easy line art colorization (mostly works, needs some testing and bug fixing)
  • Stacked brushes, redux. Our first implementation was ready in 2016, but we discarded it and will have to redo the work in a different way
  • The new text tool. This is going to be a massive simplication of our original plans… While Boudewijn is still working on making vertical text work in Qt, that’s not going to be ready. The tool itself will be simple, easy to use for the main use-cases for Krita.
  • There are some missing bits in features that are mostly done, like the Python plugin manager, or finishing up the brush editor redesign.

We are also still facing some problems on some platforms: on Linux, we don’t have a working script to package Python and GStreamer in the appimages, and the way we package G’Mic is wrong. On OSX, we don’t have a working G’Mic at all, and no support for PDF import. We do need help with that!

The current release schedule looks like this:

  • 31st December: Freeze. No new strings, no new features accepted
  • January 3rd: first Beta 1 test builds
  • January 10th: Krita 4.0 Beta 1
  • From now on, weekly development builds, if we have the resources to automate that.
  • From now on, bug fixing, bug fixing, bug fixing.
  • March 7th: first Krita 4.0 release candidate
  • March 14th: Krita 4.0

Right now, the focus is on releasing Krita 4.0, which means that currently no new Krita 3.3.x bug fix releases are planned, even though there are plenty of fixes that can be backported and should be released.

HDR Support

We’ve been approached about supporting HDR screens in Krita, which is different from the existing 10 bits/channel screens that Krita has supported on some systems for a long time. But even though we can get access to the hardware, we don’t have the time to work on it, and the API’s seem to be exclusively Windows-only.

Get Hot New Stuff

This was a 2017 Google Summer of Code project. We had an almost working implementation, but something changed server-side which means that right now nothing works. It turns out to be pretty hard to figure out what’s up, and that means it’s unlikely Krita 4.0 with have a dialog for download resources from share.krita.org.

Python API

Over the summer, we have gained a lot of experience with our Python API, also thanks to our Google Summer of Code student Eliakin. We found some places where the current API needs to be expanded and decided to do just that…

Google Summer of Code

Krita has participated in all editions of Google Summer of Code, and many of the current developers cut their teeth on Krita’s codebase during GSOC. We discussed how we could improve our participation, and how we could attract the kind of students who stay around after the project ends. One problem there is the new rules, which specify that a student can participate only twice. In our experience, it is much better for retention if students can participate during their entire university career.

We also decided to raise the bar a bit. We often see candidates who fulfill the current requirement of three patches with three one-liner patches. We’ll create a set of tasks that have a rating (1 to 3), and only candidates who have reached at least 6 points of completed tasks will be acceptable. We will also test the candidates on their ability to navigate the codebase by asking them to make a diagram of a particular subsystem.

Telemetry

The telemetry Summer of Code project has resulted in working code and a working server, but was not far enough to be mergeable to Krita. We are still interested in learning what we can about the way Krita is actually used, though. The student is interested in doing his bacherlor’s thesis on this subject, too.

Contributor Guide

Currently, information about contributing to Krita is scattered all over our website, our manual, the KDE community wiki, the source code repository and external websites. Jouni has created an outline for a contributor manual that should be part of our manual website. We’ve even already started on some parts, like the bug tracker howto!

Interview with Radian

Could you tell us something about yourself?

Hello, I’m Radian. I’m a digital artist from Russia. Honestly, I don’t know what else to say here 🙂

Do you paint professionally, as a hobby artist, or both?

I’m not working professionally yet but I’m aiming for it! I want to find work in-house because that will speed up my progress, but there is like only one serious company in my city.

What genre(s) do you work in?

I like to make neat atmospheric illustrations, usually it’s fantasy inspired works. Actually, I care more about making it look cool so I can paint whatever I like. Except cityscapes and street views. I hate those.

Whose work inspires you most — who are your role models as an artist?

I think I can name two – Mike Azevedo and Craig Mullins. I like Mike’s colors and light work so much, he definitely knows how to make it awesome. He was one of the artists who inspired me a lot from the start of my art journey.
Craig – digital art pioneer who managed to keep progressing and experimenting for so many years. I’d say I like his “philosophy”, his way to do things, his way to think.

How and when did you get to try digital painting for the first time?

About 3 years ago, I tried to make a drawing with vector tools and mouse 🙂

What makes you choose digital over traditional painting?

Colors, all of them! It’s easy to mess with, easy to experiment. I like good traditional art, I like its aesthetics, but I don’t like working with traditional media, mixing colors, cleaning brushes and such.

How did you find out about Krita?

From a comment on some entertainment site. It was something like “hey, there is also Krita, it can do the same things as SAI and even more, plus it’s free”.

What was your first impression?

“Whoa, so many buttons! I’m gonna press them all”

What do you love about Krita?

Someone can say that they like blending brushes or a lot of tools or something else. But I’ll say Krita is extremely comfy. It’s very logical (in most areas 🙂 and it has many little but very handy things.

What do you think needs improvement in Krita? Is there anything that really annoys you?

There are some things that slightly annoy me but those are either very hard to fix or super small and not even worth ranting about. Sometimes both. Improvements? I’m sort of an idea guy so I can write a kilometer-long text about what can be improved. Though I can’t say anything needs to be improved in Krita. Vectors and text tools aren’t good but they’re already in development.

What sets Krita apart from the other tools that you use?

The best balance between comfort and functionality. There are a few cool Photoshop things that I’d like to have in Krita but if I start Photoshop I
can count a lot of things there that I’d like to get from Krita. It’s just hard to use for me. Any other program I tried could win one or two rounds, but Krita always had more. More functions, more tricks for artists, more little handy things and more possibilities. Also I think Python scripting will add a new level of comfort that will never be defeated.

If you had to pick one favourite of all your work done in Krita so far, what would it be, and why?

I tend to hate any of my artwork if it is more than 1-3 months old but there are a couple of exceptions. The Kiki painting I made for the artbook “Made with Krita” is one of them. I used a bunch of new tricks in here and probably made a few good choices by accident.

What techniques and brushes did you use in it?

I used my own brush pack, many brushes “inspired” by some Photoshop brushpack from pro artists. There was’t a single brush that satisfied me so I had to learn brush creation magic 🙂

As I said, I like Mullins’s philosophy: more looking, more thinking, less painting. I usually trying to do as much as I can with as few strokes as I can. But I like to experiment as well. If there are so many techniques, why should I use only one? Try everything, break all rules! Learn from mistakes and especially learn from successful experiments.

Where can people see more of your work?

http://radian-art.tumblr.com/
https://radian1.deviantart.com/
https://www.artstation.com/radan

Anything else you’d like to share?

It’s kinda funny to see how such a small team makes such big progress in development while Photoshop stagnates for years with a big company full of professionals.

Thanks for all your work guys, keep it up!

November 26, 2017

Reading an IR Remote on a Raspberry Pi Stretch with LIRC

I wrote earlier about how to use an IR remote on Raspbian Jessie.

It turns out several things have changed under Raspbian Stretch. Here's the abbreviated procedure for Stretch:

Install LIRC

$ sudo apt-get install lirc

Enable the LIRC Overlay

Eedit /boot/config.txt as root, look for this line and uncomment it:

# Uncomment this to enable the lirc-rpi module
dtoverlay=lirc-rpi
Or if you prefer to use a pin other than 18, change the pin assignment like this:
# Uncomment this to enable the lirc-rpi module
dtoverlay=lirc-rpi,gpio_in_pin=25,gpio_out_pin=17

See /boot/overlays/README for more information on overlays.

Fix the LIRC Options

Edit /etc/lirc/lirc_options.conf, comment out the existing driver and device lines, and add:

driver    = default
device = /dev/lirc0

Reboot and stop the daemon

Reboot the Pi.

Now a bunch of LIRC daemons will be running. You don't want them while you're configuring, and if you're eventually going to be reading button presses from Python, you don't want them at all.

Disable them temporarily with

sudo systemctl stop lircd
which seems to be shorthand for
sudo systemctl stop lircd.socket
sudo systemctl stop lircd.service

Be sure to check with ps aux | grep lirc to make sure you've turned them off.

If you want to disable them permanently,

sudo systemctl disable lircd.socket lircd.service lircd-setup.service lircd-uinput.service lircmd.service
I got that list from:
systemctl list-unit-files | grep lirc

But you need them if you want to read from the /var/run/lirc/lircd socket.

Use mode2 to verify it sees the buttons

With the daemons not running, a program called mode2 can verify that your device's buttons are being seen at all. I have no idea why it's named that, or what Mode 1 is.
mode2 -d /dev/lirc0

You should see lots of output. If you don't, double-check your wiring and everything else you've done up to now.

Set up an lircd.conf

Here's where it gets difficult. On Jessie, you could run irrecord -d /dev/lirc0 ~/lircd.conf as described in my earlier article.

However, that doesn't work on stretch. There's apparently a bug in the irrecord in stretch that makes it generate a file that doesn't work. If you try it and it doesn't work, run tail -f /var/log/messages | grep lirc and you may see Info: Cannot configure the rc device for /dev/lirc0 and when you press buttons you'll see Notice: repeat code without last_code received but you won't get any keys.

If you have a working lirc setup from a Jessie machine, try it first. If it doesn't work, there's a script you can try that converts older lirc conf files to a newer format. The safest way to try it is to copy (with cp -a) the whole /etc/lirc directory to a local directory and run:

/usr/share/lirc/lirc-old2new your-local-copy
Or if you feel brave, back up /etc/lirc and run sudo /usr/share/lirc/lirc-old2new with no arguments. Either way, you should get an lirc.conf that has a chance of working with stretch.

If you don't have a working Jessie config, you're in trouble. You might be able to edit the one from irrecord to make it work. Here's the first part of my working Jessie lircd.conf:

begin remote

  name  /home/pi/lircd.conf
  bits           16
  flags SPACE_ENC|CONST_LENGTH
  eps            30
  aeps          100

  header       9117  4494
  one           569  1703
  zero          569   568
  ptrail        575
  repeat       9110  2225
  pre_data_bits   16
  pre_data       0xFD
  gap          108337
  toggle_bit_mask 0x0

      begin codes
          KEY_POWER                0x00FF
          KEY_VOLUMEUP             0x807F
          KEY_STOP                 0x40BF
          KEY_BACK                 0x20DF
          KEY_PLAYPAUSE            0xA05F
          KEY_FORWARD              0x609F
          KEY_DOWN                 0x10EF
and here's the corresponding part of the nonworking one generated on Stretch:
begin remote

  name  DingMai
  bits           32
  flags SPACE_ENC|CONST_LENGTH
  eps            30
  aeps          100

  header       9117  4494
  one           569  1703
  zero          569   568
  ptrail        575
  repeat       9110  2225
  gap          108337
  toggle_bit_mask 0x0
  frequency    38000

      begin codes
          KEY_POWER                0x00FD00FF 0xBED8F1BC
          KEY_VOLUMEUP             0x00FD807F 0xBED8F1BC
          KEY_STOP                 0x00FD40BF 0xBED8F1BC
          KEY_BACK                 0x00FD20DF 0xBED8F1BC
          KEY_PLAYPAUSE            0x00FDA05F 0xBED8F1BC
          KEY_FORWARD              0x00FD609F 0xBED8F1BC
          KEY_DOWN                 0x00FD10EF 0xBED8F1BC

It looks like setting bits to 16 and then using the second quartet from each key might work. So try that if you're stuck.

Once you get irw working, you're home free. The Python modules probably still won't do anything useful, but you can use my pyirw.py script as a model for a simple way to read keys from the lirc daemon.

In case you hit problems beyond what I saw, I found this discussion useful, which links to a complete GitHub gist of instructions for setting up lirc on Stretch. Those instructions have a couple of extra steps involving module loading that it turned out I didn't need, and on the other hand it doesn't address the problems I saw with irrecord. It looks like lirc configuration is a black art, not a science. See what works for you. Good luck!

November 25, 2017

Go Halifax Explosion!

Heard on CBC this morning that one of many names floated for a possible Halifax CFL expansion team is the Halifax Explosion.

Brilliant? Inappropriate? I think both?

November 24, 2017

The Hard Truths of Canada’s History

Speaking of the victims of residential schools in Newfoundland & Labrador, The Prime Minister of Canada said:

“Many were sorely neglected and not properly fed, clothed, or housed. Others suffered physical, psychological, and sexual abuse. All were deprived of the love and care of their families, of their parents, and of their communities. These are the hard truths that are part of Canada’s history.

Lest it be seen as a partisan issue, the previous Prime Minister, Stephen Harper, said in 2008:

“Two primary objectives of the residential school system were to remove and isolate children from the influence of their homes, families, traditions and cultures, and to assimilate them into the dominant culture. These objectives were based on the assumption Aboriginal cultures and spiritual beliefs were inferior and unequal. Indeed, some sought, as it was infamously said, “to kill the Indian in the child.” Today, we recognize that this policy of assimilation was wrong, has caused great harm, and has no place in our country.”

It may just be words, but it is important that these issues are acknowledged by the highest office in our country.

The greatest feature of the Web may be that it ignores double spaces after a period.

November 22, 2017

Giving Thanks


Giving Thanks

For a wonderful community

This is becoming a sort of tradition for me to post something giving thanks around this holiday. I think it’s because this community has become such a large part of my life (even if I don’t have nearly as much time to spend on it as I’d like). Also, I think it helps to remind ourselves once in a while of the good things that happen to us. So in that spirit…

Financial Supporters

I want to start things off by acknowledging those that go the extra mile and help offset the costs of the infrastructure to keep this crazy ship afloat (sorry, I’m an ocean engineer by training and a sailor - so nautical metaphors abound!).

Holy Benefactors, Batman!

Once again the amazing Dimitrios Psychogios has graciously covered our server expenses (and then some) for another full year. On behalf of the community, and particularly myself, thank you so much! Your generosity will cover infrastructure costs for the year and give us room to grow as the community does.

We also have some awesome folks who support us through monthly donations (which are nice because we can plan better if we need to). Together they cover the costs of data storage + transfer in/out of Amazon AWS S3 storage (basically the storage and transfer of all of the attachments and files in the forums). So thank you, you cool froods, you really know where your towels are:

Thank you all! If you happen to see any of these great folks around the forum consider taking a moment to thank them for their generosity! If you’d like to join them in supporting the site financially, check out the support page.

Growth

The community has just been amazing, and we’ve seen nice growth this past year. Since the end of August we’ve seen about a 50% increase in weekly sessions on discuss. We’re currently hovering around 2,500 daily pageviews on the forums:

PIXLS.US discuss traffic

We’ve added almost 950 new users, or almost 3 new users every day!

There have been quite a few interesting discussions happening on the forums as well. The RawTherapee folks have some neat conversations going on (Local Lab build, New Windows builds, and Pixel Shift!), and @Carmelo_DrRaw (creator of the PhotoFlow editor) has been packaging a GIMP 2.9.X AppImage as well!

Of course, the fun news for many was @houz finally pushing out a Windows version of darktable that was made possible through the help of Peter Budai.

raw.pixls.us

I figure @LebedevRI will yell at me if I forget to mention raw.pixls.us (RPU) again. Back in January @andabata built a new site to help pick up the work of the old rawsamples.ch website to collect raw sample files for testing.

So thank you @andabata and @LebedevRI for your work on this! A big thank you to everyone who has taken the time to check the site and upload missing (or non-freely licensed) raw files to include!

While we’re talking about RPU, please consider having a look at this post about it on discuss and take a few minutes to see if you might be able to contribute by providing raw samples that we are missing or need (see the post for more details). If you don’t have something we need, please consider sharing the post on social media to help us raise awareness of RPU! Thank you!

digiKam

If you’re not aware of it, one of the things we try to do here beside run the site and forum is to assist projects with websites and design work if they want it. Earlier this year the digiKam team needed to migrate their old Drupal website to something more modern (and secure) and @paperdigits figured, “why not”?

digiKam Logo

So we rolled up our sleeves and got them setup with a newly designed static website built using Hugo (which was completely new to me). We were also able to manage their comments on the website for them by embedding topics from right here on discuss. This way their users can still own their comments and we can manage spam and moderate things for them.

The best part, though, is the addition of their users and knowledge to the community!

darix

I want to personally take a moment to thank @darix for all the work he does keeping things running smoothly here. If you don’t see him, it means all the work he’s doing is paying off.

I speak with him daily and see firsthand the great work he’s doing to make sure all of us have a nice place to call home. Thank you so much, @darix!

Mica

As usual @paperdigits (https://silentumbrella.com) also has a great attitude and pro-active approach to the community which I am super thankful for. He also does things that aren’t always visible, but are essential to keeping things running smoothly, like moderating the forum, checking the health of sites we are helping to manage, and writing/editing posts.

I can’t stress enough how much it helps to keep your interest and spirits engaged in the community when you have someone else around who’s so positive and helpful. Thank you so much, @paperdigits!

All of You

At the end of the day this is a community, and it’s vibrancy and health is a direct result of all of you, its members. So above all else this is by far the thing I am most thankful for - getting to meet, learn, and interact with all of you.

November 21, 2017

local laplacian pyramids

improving contrast with the local laplacian filter

sometimes difficult lighting situations arise which, when taking photographs, result in unappealing pictures. for instance very uniform lighting on a cloudy day may give dull results, while very contrasty illumination (such as back lit) may require to compress the contrast to embrace both highlights and shadows in the limited dynamic range of the output device.

refer to the following two shots as examples:

  • input + high contrast b/w
  • hdr + compressed

many options to achieve this exist in literature and many of them are implemented in darktable already.

this post is about yet another approach to this which turned out to be extremely versatile, almost artifact-free, and reasonably fast to compute: the local laplacian pyramid.

repeat figure with curves and then it’s pretty much exposure fusion with many different images (one per gray level)

local contrast with the local laplacian pyramid

vanilla laplacian pyramids [0] are known to be a good tool to blend over between two images. for their applicability to local contrast, not so much. perhaps surprisingly, following up on exposure fusion (we blogged about that for darktable before ), a clever way to modify local contrast based on laplacian pyramids has been devised [1]: it works by pretty much creating a separate laplacian pyramid for every pixel in the image, and then selecting coefficients from these these based on certain criteria.

luckily, this can be reduced to processing the image only a small, fixed number of times [2]. the procedure is then basically:

  • process the image n times, mapping it through a curve (see below for examples)
  • create the n laplacian pyramids that go with the images
  • merge into a final laplacian pyramid
  • collapse this output pyramid to create the output image.

this is nicely illustrated in this video exported from halide:

also, as it turns out, the gpu is really good at processing laplacian pyramids. the opencl port of this turned out to be very useful.

the mapping function

why can we get away with this small fixed number n of processed pyramids? [2] gives the explanation in sec. 3:

If r is band-limited,

(r is the family of curves which is applied to the input image).

now if you look at the left image (which is taken from fig. 6 in [1]), it has clear kinks where the center part joints the straight lines. not band limited at all. don’t use this curve at home. it will produce random aliasing when used with the fast local laplacian code.

fig 6   clarity

darktable uses what is shown on the right instead: the contrast-s curve in the center part is modelled by a derivative of gaussian (with infinite support), which is added to the straight lines on either side, which are blended over using a quadratic bezier curve. you can look up the precise code if you’re interested.

now changing the contrast-s curve in the middle changes local contrast, that’s great. looking at the sides of this curve, we would immediately want to use it for shadow lifting and highlight compression too. in fact that’s possible (see following section with images and accompanying curves). unfortunately some of the proposed optimisations in the original papers can’t be applied any more. for the shadow/highlight use case, the pyramid needs to be constructed all the way all the time, we can’t stop after three levels (or else the shadow lifting would depend on the scale of the image).

this means darktable will always build the full pyramids, making this a little slower, but yielding best results.

the darktable ui

you can find our implementation of this filter in the local contrast module, when switched to local laplacian mode:

ui

it allows you to change detail (or clarity or local contrast), highlight contrast, and shadow contrast separately. the last slider governs how wide the central region of the curve is, i.e. what is classified as shadow vs. mid-tone.

in the following, we’ll go through these contrast sliders, showing a couple of corresponding conversion curves and the resulting effect on the output image.

drag the separator in the middle to compare before/after!

increased local contrast:

ident   clarity

pulled highlights down:

yes, the curve changes the bottom part. which you would think belongs to the shadows. turns out it doesn’t.

ident   highlights

pushed shadows up:

ident   shadows

inverse of the above

and the inverse is also possible, top to bottom, left to right: original, removed details, pushed highlights out even more, pulled shadows deeper.

application to hdr compression

in the following there are some comparison images from the previous post on exposure fusion. these have been processed again with a different approach:

  • to compress the dynamic range, a very flat base curve has been used which maxes out at 0.5 for input values of 1.0.
  • as contrast-s curve, the tone curve module has been used, in rgb mode for automatic colour saturation compensation.
  • the curve was created to match an out of camera raw+jpg pair using darktable-chart which also created a clut for the colour lookup table module.

this approach will leave the image flat and dull (due to the flat base curve). to add back additional detail, we use the local laplacian module. this turns out to work a lot better together with a contrast-s tone curve than with a contrast-s base curve, since the base curve comes before the local contrast module in the pipeline, while the tone curve comes after. this way, the local contrast module receives more linear input.

in fact, sometimes a contrast-s curve is not necessary at all for hdr input. exposure fusion shows some nicer behaviour with respect to colour saturation at times, but sometimes produces halo like artifacts. i find it easier to create good results with the local laplacian when aiming for a low key look (see the definition in the darker clouds in the left of the first image). really up to you and any given input images which method works better.

example history stacks are in the example comparison images seen below: 0 1 2.

application to high contrast monochrome

a note on noise

when increasing contrast, you want to make sure the input isn’t noisy, or else that noise would be accentuated even more. but even when not increasing contrast, the local laplacian filter will show structured artifacts for noisy input. i find this mostly goes away when enabling at least chroma denoising for instance in the profiled denoising module.

literature

What’s important counts

In the extraordinary documentary, The Vietnam War by Ken Burns & Lynn Novick, an ‘Army Advisor’ named James Willbanks says:

“If you can’t count what’s important, you make what you can count important.”

This is true in more than war.

November 20, 2017

LVFS Article On OpenSource.com

I wrote an article on the LVFS for OpenSource.com. If you’re interested in an overview of how firmware updates work in Linux, and a little history it might be an interesting read.

November 16, 2017

Talking picture in picture

Tech Youtube-guy extraordinaire, MKBDH, pulls off a subtle but clever way to show how video playback works on the new iPhone X screen (relevant portion is around 5:50).

November 15, 2017

security things in Linux v4.14

Previously: v4.13.

Linux kernel v4.14 was released this last Sunday, and there’s a bunch of security things I think are interesting:

vmapped kernel stack on arm64
Similar to the same feature on x86, Mark Rutland and Ard Biesheuvel implemented CONFIG_VMAP_STACK for arm64, which moves the kernel stack to an isolated and guard-paged vmap area. With traditional stacks, there were two major risks when exhausting the stack: overwriting the thread_info structure (which contained the addr_limit field which is checked during copy_to/from_user()), and overwriting neighboring stacks (or other things allocated next to the stack). While arm64 previously moved its thread_info off the stack to deal with the former issue, this vmap change adds the last bit of protection by nature of the vmap guard pages. If the kernel tries to write past the end of the stack, it will hit the guard page and fault. (Testing for this is now possible via LKDTM’s STACK_GUARD_PAGE_LEADING/TRAILING tests.)

One aspect of the guard page protection that will need further attention (on all architectures) is that if the stack grew because of a giant Variable Length Array on the stack (effectively an implicit alloca() call), it might be possible to jump over the guard page entirely (as seen in the userspace Stack Clash attacks). Thankfully the use of VLAs is rare in the kernel. In the future, hopefully we’ll see the addition of PaX/grsecurity’s STACKLEAK plugin which, in addition to its primary purpose of clearing the kernel stack on return to userspace, makes sure stack expansion cannot skip over guard pages. This “stack probing” ability will likely also become directly available from the compiler as well.

set_fs() balance checking
Related to the addr_limit field mentioned above, another class of bug is finding a way to force the kernel into accidentally leaving addr_limit open to kernel memory through an unbalanced call to set_fs(). In some areas of the kernel, in order to reuse userspace routines (usually VFS or compat related), code will do something like: set_fs(KERNEL_DS); ...some code here...; set_fs(USER_DS);. When the USER_DS call goes missing (usually due to a buggy error path or exception), subsequent system calls can suddenly start writing into kernel memory via copy_to_user (where the “to user” really means “within the addr_limit range”).

Thomas Garnier implemented USER_DS checking at syscall exit time for x86, arm, and arm64. This means that a broken set_fs() setting will not extend beyond the buggy syscall that fails to set it back to USER_DS. Additionally, as part of the discussion on the best way to deal with this feature, Christoph Hellwig and Al Viro (and others) have been making extensive changes to avoid the need for set_fs() being used at all, which should greatly reduce the number of places where it might be possible to introduce such a bug in the future.

SLUB freelist hardening
A common class of heap attacks is overwriting the freelist pointers stored inline in the unallocated SLUB cache objects. PaX/grsecurity developed an inexpensive defense that XORs the freelist pointer with a global random value (and the storage address). Daniel Micay improved on this by using a per-cache random value, and I refactored the code a bit more. The resulting feature, enabled with CONFIG_SLAB_FREELIST_HARDENED, makes freelist pointer overwrites very hard to exploit unless an attacker has found a way to expose both the random value and the pointer location. This should render blind heap overflow bugs much more difficult to exploit.

Additionally, Alexander Popov implemented a simple double-free defense, similar to the “fasttop” check in the GNU C library, which will catch sequential free()s of the same pointer. (And has already uncovered a bug.)

Future work would be to provide similar metadata protections to the SLAB allocator (though SLAB doesn’t store its freelist within the individual unused objects, so it has a different set of exposures compared to SLUB).

setuid-exec stack limitation
Continuing the various additional defenses to protect against future problems related to userspace memory layout manipulation (as shown most recently in the Stack Clash attacks), I implemented an 8MiB stack limit for privileged (i.e. setuid) execs, inspired by a similar protection in grsecurity, after reworking the secureexec handling by LSMs. This complements the unconditional limit to the size of exec arguments that landed in v4.13.

randstruct automatic struct selection
While the bulk of the port of the randstruct gcc plugin from grsecurity landed in v4.13, the last of the work needed to enable automatic struct selection landed in v4.14. This means that the coverage of randomized structures, via CONFIG_GCC_PLUGIN_RANDSTRUCT, now includes one of the major targets of exploits: function pointer structures. Without knowing the build-randomized location of a callback pointer an attacker needs to overwrite in a structure, exploits become much less reliable.

structleak passed-by-reference variable initialization
Ard Biesheuvel enhanced the structleak gcc plugin to initialize all variables on the stack that are passed by reference when built with CONFIG_GCC_PLUGIN_STRUCTLEAK_BYREF_ALL. Normally the compiler will yell if a variable is used before being initialized, but it silences this warning if the variable’s address is passed into a function call first, as it has no way to tell if the function did actually initialize the contents. So the plugin now zero-initializes such variables (if they hadn’t already been initialized) before the function call that takes their address. Enabling this feature has a small performance impact, but solves many stack content exposure flaws. (In fact at least one such flaw reported during the v4.15 development cycle was mitigated by this plugin.)

improved boot entropy
Laura Abbott and Daniel Micay improved early boot entropy available to the stack protector by both moving the stack protector setup later in the boot, and including the kernel command line in boot entropy collection (since with some devices it changes on each boot).

eBPF JIT for 32-bit ARM
The ARM BPF JIT had been around a while, but it didn’t support eBPF (and, as a result, did not provide constant value blinding, which meant it was exposed to being used by an attacker to build arbitrary machine code with BPF constant values). Shubham Bansal spent a bunch of time building a full eBPF JIT for 32-bit ARM which both speeds up eBPF and brings it up to date on JIT exploit defenses in the kernel.

seccomp improvements
Tyler Hicks addressed a long-standing deficiency in how seccomp could log action results. In addition to creating a way to mark a specific seccomp filter as needing to be logged with SECCOMP_FILTER_FLAG_LOG, he added a new action result, SECCOMP_RET_LOG. With these changes in place, it should be much easier for developers to inspect the results of seccomp filters, and for process launchers to generate logs for their child processes operating under a seccomp filter.

Additionally, I finally found a way to implement an often-requested feature for seccomp, which was to kill an entire process instead of just the offending thread. This was done by creating the SECCOMP_RET_ACTION_FULL mask (née SECCOMP_RET_ACTION) and implementing SECCOMP_RET_KILL_PROCESS.

That’s it for now; please let me know if I missed anything. The v4.15 merge window is now open!

© 2017 – 2018, Kees Cook. This work is licensed under a Creative Commons Attribution-ShareAlike 3.0 License.
Creative Commons License

November 14, 2017

Firefox is (still) great

Perhaps due to my history with Mozilla and Firefox, I’ve been a happy Firefox user since version 0.6 (14 years ago!?). In recent years, the Chrome browser from Google has become the most commonly used browser, especially among web developers.

I’m delighted to see that Mozilla has made great strides in improving Firefox and is winning people over again. The latest version of Firefox, released today, is worth a try. There are also some great nerdy details on how Firefox has improved.

Even if Firefox doesn’t regain the market-share it once had, these efforts push the other browser vendors to improve and generally improve the Web as a platform.

Thanks and congrats to all of the designers, engineers, and other humans who continue to improve Firefox. The update icon looks slick too.

November 13, 2017

Interview with Lars Pontoppidan

Could you tell us something about yourself?

Yes certainly! I’m Lars Pontoppidan; a 36 year old, self-employed programmer, game developer, musician and artist.

I’ve been drawing and painting since I could put my pen to the paper – so about 35 years.

I made my first recognizable painting when I was around 3 or 4 – my mom still has it framed at her house.

I’ve always wanted to end up at some level where I could combine all my skills and hobbies. Somewhere along the way I found out that game development demand a lot of the skills I possess – so 1.5 years ago, I decided to cancel all my contracts with my clients and go for a new path in life as “indie game developer”. I’ve now found out that it’s probably the worst time I could ever get into indie game development. The bubble has more or less already burst. There’s simply too many game releases for the consumers to cope with at the moment. But hey I’ve tried worse so it doesn’t really bother me – and I get to make art with Krita!

Do you paint professionally, as a hobby artist, or both?

Both I’d say. I’ve always been creating things on a hobby level – but have also delivered a lot of designs, logos and custom graphics as self-employed. I like the hobby work the most – as there are no deadlines or rules for when the project is done.

What genre(s) do you work in?

Cartooning, Digital painting, Animation and Video game art. All these (and maybe more) blend in when producing a game. I also like painting dark and gloomy pictures once in a while.

I think I’ve mostly done cartoon styled work – but with a grain of realism in it. My own little mixture.

I started out with pencil and paper – moved to the Deluxe Paint series, when I got my first Amiga – and ended up with Krita (which is an absolute delight to work with. Thanks to you guys!). I still occasionally do some sketching with pencil and paper – depending on my mood.

Whose work inspires you most — who are your role models as an artist?

* A list too long for me to compile here, of sci-fi fantasy artists. Peter Elson is the first that comes to mind. These artists, in my opinion, lay the very foundation of what’s (supposedly) possible with human technology – and currently, the only possibility to get a glimpse of how life might look like other places in the vast universe that surrounds us. It’s mind blowing how they come up with all the alien designs they do.

* Salvador Dalí – It’s hard to find the right words for his creations – which, I think, is why his works speak to me.

* “vergvoktre” He’s made some really dark, twisted and creepy creations that somehow get under my skin.

How and when did you get to try digital painting for the first time?

The very first digital painting program I’ve ever tried was KoalaPainter for the Commodore 64. I had nothing but a joystick and made, if I recall correctly, a smiley face in black and white.

Thankfully my Amiga 500 came with a copy of Deluxe Paint IV, a two-button mouse and the luxury of a 256+ color palette.

What makes you choose digital over traditional painting?

The glorious “Undo” buffer. I mean… It’s just magic.Especially in the first part of the day (before the first two cups of coffee) where your hand just won’t draw perfect circles, nor any straight lines.

How did you find out about Krita?

I read an article about the Calligra office suite online. It described how Calligra compared to Open Office. I eventually installed it to see how it compared to Open Office and boom there was Krita as part of the package. This was my first encounter – unfortunately it ended up with an uninstall – because of stability issues with the Calligra suite in general.

What was your first impression?

The first impression was actually really good – unfortunately it ended up a bit in the shadows of the Calligra suite’s combined impression. This wasn’t so positive after a few segfaults in the different applications. Luckily I tried Krita later when it entered the Qt5 based versions. I haven’t looked back since.

What do you love about Krita?

The brush engines and the “Layers” docker.

The brushes, and most of the default settings for them, just feel right. Also the many options to tweak the brushes are really awesome.

The layers docker was actually what gave me the best impression of the program – you had working group layers – and you could give any layer the same names! None of the graphic creation applications I used a few years back had these basic, fundamental features done right (Inkscape and GIMP – I’m looking at you). Krita’s layers didn’t feel somewhat broken, hacked-on and had no naming scheme limitations. A small thing that has made a big difference to me.

What do you think needs improvement in Krita? Is there anything that really annoys you?

Uhm… I was going to write ‘speed’ – but everybody is screaming for more of that already. I know how the developers are doing their best to get more juice.

Some great overall stability would be nice. I’ve only ever had 2 or 3 crashes with GIMP over a long period of time – the count is a bit higher with Krita – on a shorter time scale.

My biggest feature request would be: Cut’n’paste functionality through multiple layers, that also paste in separate layers. This would greatly improve my workflow. I’ve always worked with a group layer containing separate layers for outline, color, texture, shadow etc. – on each e.g. movable part in a character rig. So I would really benefit from a (selection based) cut’n’paste that could cut through all the selected layers – and paste all these separate selection+layers elsewhere in the layer tree.

What sets Krita apart from the other tools that you use?

I find that most of Krita’s tools actually do what you expect them to do – without any weird limitations or special cases. Plus the different brushes, brush engines and all the flexibility to tweak them, are real killer features.

The non-destructive masks (Transparency, Filter and Transform) are also on my list of favourite features. I use these layer types a lot when creating game art – to make them blend in better with the game backgrounds.

And maybe the single most important thing: it’s free and open source. So I’m quite certain I will be able to open up my old Krita files many years into the future.

… and speaking of the future; I really look forward to getting my hands dirty with the Python scripting API.

If you had to pick one favourite of all your work done in Krita so far, what would it be, and why?

It would have to be the opening scene of my upcoming 2D game “non”. It’s using a great variety of Krita’s really awesome and powerful features. The scenes in the game features full day and night cycles where all the lighting and shadows change dynamically – this makes it especially hard to get beautiful painted scenes in all the states each scene has between day and night. Krita’s tool set makes it easier and quicker for me to test out a specific feature for an object or sprite – before throwing it into the game engine.

The biggest scene I have so far is 10200×4080 pixels – Krita was actually performing decently up to a certain point where I had to break the scene into smaller projects. I’m not blaming Krita for this 🙂

What techniques and brushes did you use in it?

For cartoon styled work I use a Group layer containing:
* background blend (Transparency Mask)
* shadows (Paint layer)
* outlines (Paint layer)
* textures (Group layer)
* solid base color(s) (Paint layer)

For outlines I use the standard Pixel brush ‘Ink_gpen_10’ – it has a really nice sharp edge at small tip sizes. For texturing I mostly use the ‘Splatter_thin’ Pixel brush – with both standard and custom brush tips and settings depending on the project at hand. For shadowing I really like the ‘Airbrush_pressure’ and ‘Airbrush_linear_noisy’ Pixel brushes. I use a selection mask based on the solid base color layer (Layer name -> Right mouse click -> Select Opaque) – and start shadowing the object.

Where can people see more of your work?

In my games: http://games.blackgrain.dk/
On my band album covers: https://barricadecph.bandcamp.com/

Anything else you’d like to share?

I’d like to thank everyone involved with Krita for making this great open source and free software available to the world. I hope to soon get enough time on my hands to help the project grow.

Take care and be nice to each other.

Selling the Flu Shot

Every time I see a news story about the flu shot (which is available for free on Prince Edward Island this year), there’s always a stock photo close-up of a needle jabbing into an arm.

If you want to encourage people to get the flu shot, don’t use a photo of the one thing people don’t like about the flu shot.

I was able to get the flu shot for free without an appointment (or any wait time) at Shoppers Drug Mart.

November 09, 2017

Learn Digital Painting with Krita in Bogota, Colombia

Lina Porras and David Saenz from the Ubuntu Colombia user group wrote to tell us that they will give an introduction to digital painting with Krita starting this Saturday. David will be teaching Krita four Saturday sessions.

It will be an introductory course where people from 14 years old and older will learn the basics of digital painting and will start painting in Krita. Here is more information:

http://ubuntu-co.com/2017/10/09/primer-curso-de-krita

And you can follow them on twitter (@ubuntco and facebook as well: https://www.facebook.com/UbuntuColombia

If you are thinking of organizing a Krita course for your local user group, community art college or similar,  contact us so we can help you spread the word, too!

November 08, 2017

The Zen Microwave

Here’s a free invention idea for you (in that I have a stupid idea and will not do anything with it): The Zen Microwave.

The Zen Microwave only has one control: an on/off switch. Once you’ve started it, you have to remember to turn it off or your left-overs will turn into exploding spaghetti-charcoal.

If you want to heat up your lunch, it can go one of two ways:

  1. Put your lunch in the microwave and turn it on (remember, there’s no timer)
  2. Stand there for two minutes, focused and present
  3. Turn the microwave off and enjoy your hot lunch

Or:

  1. Put your lunch in the microwave and turn it on
  2. Wander around the kitchen, get a glass of water, check in on your online click-farm business on your phone, thumb through the Home Hampers & Hobbits flyer
  3. Smell burning and notice that your lunch has been super-heated for 10 minutes

Is it dangerous? Yes – but how else are you ever going to learn?

November 07, 2017

Hardware CI Tests in fwupd

Usually near the end of the process of getting a vendor on the LVFS I normally ask them to send me hardware for the tests. Once we’ve got a pretty good idea that the hardware update process is going to work with fwupd (i.e. they’re not insisting on some static linked ELF to be run…) and when they’ve got legal approval to upload the firmware to the LVFS (without an eyewateringly long EULA) we start thinking about how to test the hardware. Once we say “Product Foo from Vendor Bar is supported in Linux” we better make damn sure it doesn’t regress when something in the kernel changes or when someone refactors a plugin to support a different variant of a protocol.

To make this task a little more manageable, we have a little python script that helps automate the devices that can be persuaded to enter DFU mode themselves. To avoid chaos, I also have a little cardboard tray under a little HP Microserver with two 10-port USB hubs with everything organised. Who knew paper-craft would be such an important skill at Red Hat…

As the astute might notice, much of the hardware is a bare PCB. I don’t actually need the complete device for testing, and much of the donated hardware is actually a user return or with a cosmetic defect, or even just a pre-release PCB without the actual hardware attached. This is fine, and actually preferable to the entire device – I only have a small office!

As much of the hardware needs special handling to put it in update mode we can’t 100% automate this task, and sometimes it really is just me sitting in front of the laptop pressing and holding buttons for 30 minutes before uploading a tarball, but it’s sure it comforting to know that firmware updates are tested like this. As usual, thanks should be directed to Red Hat for letting me work on this kind of stuff, they really are a marvelous company to work for.

November 06, 2017

4.0 Development Update

And then we realized we hadn’t posted news about ongoing Krita development for some time now. The main reason is that we’ve, well, been really busy doing development. The other reason is that we’re stuck making fully-featured preview builds on OSX and Linux. More about that later…

So, what’s been going on? Some of the things we’ve been doing were backported to Krita 3.2 and 3.3, like support for the Windows 8 Pointer API, support for the ANGLE Direct3D display renderer, the new gmic-qt G’Mic plugin, new commandline options, support for touch painting, the new smart patch tool, new brush presets and blending modes… But there is also a lot of other work that simply couldn’t be backported to 3.x.

The last time we did a development update with Krita 4.0 was in June 2017: the first development build for 4.0 already had a large number of new features:

  • the SVG-based vector layers with improved vector handling tools,
  • Allan Marshall’s new airbrush system,
  • Eugene Ingerman’s healing brush,
  • a new export system that reports which parts of your image cannot be saved to your chosen file format: and that is now improved: saving now happens in the background. You can press save, and continue painting. Autosave also doesn’t interrupt your painting anymore.
  • Wolthera’s new and improved palette docker
  • A new docker for loading SVG symbol collections, Which now comes with a new symbol libary with brush preset icons. Perfect with the new brush editor.
  • We added Python scripting (only available in the Windows builds: we need platform maintainers). Eliakin and Wolthera have spent the summer adding great new python-based plugins, extending and improving the scripting API while working:
    • Ten brushes: a script to assign ten favorite brushes to hotkeys
    • Quick settings docker: with brush size, opacity and flow
    • Comic Projects Management tools
    • And much, much more

What has been recently added to Krita 4.0

Big performance improvements

After the development build release we sent out a user survey:  In case you didn’t see the results of our last survey this was the summary.

The biggest item on the list was lag. Lag can have many meanings, and there will always be brushes or operations that are not instant. But we had the opportunity the past couple of months to work on an outside project to help improve the performance of Krita. While we knew this might delay the release of Krita 4.0, it would be much appreciated by artists. Some of the performance improvements contain the following:

  • multi-threaded performance using all your CPUs for pixel brush engines (80% of all the brushes that are made).
  • A lot of speed optimizations with dab grouping for all brushes
  • more caching to speed up brush rendering across all brushes.

Here’s a video of Wolthera using the multithreaded brushes:

Performance Benchmarking

We also added performance benchmarking. We can see much more accurately how brushes are performing and make our brushes better/optimized in the future:

 

 

Pixel Grid

Andrey Kamakin added an option to show a thin grid around pixels if you zoom in enough:

Live Brush Preview

Scott Petrovic has been working with a number of artists to rework the brush editor. There have been many things changed including renaming brushes and  better saving options.  There’s also a live stroke preview now to see what happens when you change settings. Parts of the editor can be shown or hidden to accommodate for smaller monitors.

Isometric Grid

The grid now has a new Isometric option. This can be controlled and modified through the grid docker:

Filters

  • A new edge detection filter
  • Height to normal map filter
  • Improved gradient map filter
  • A new ASC-CDL color balance filter with slope, offset and power parameters

Layers

  • File layers now can have the location of their reference changed.
  • A convert layer to file layer option has been added that saves out layers and replaces them with a file layer referencing them.

Dockers

  • A new docker for use on touch screens: big buttons in a layout that resembles the button row of a Wacom tablet.

More

And there’s of course a lot of bug fixes, UI polish, performance improvements, small feature improvements. The list is too long to keep here, so we’re working on a separate release notes page. These notes, like this Krita 4.0 build, are very much a work in progress!

Features currently working on

There are still a number of features we want to have done before we release Krita 4.0:

  • a new text tool (we have started the ground work for this, but it still needs a lot more work)
  • a faster colorize mask tool (we need to make this much faster as it is currently too slow)
  • stacked brushes where you can have multiple brush tips similar to other applications.

And then there are no doubt things missing from the big new features, like SVG vector layers and Python scripting that need to be implemented, there will be bugs that need to be fixed. We’ve made packages for you to download and test, but be warned, there are bugs. And:

This is pre-alpha code. It will crash. It will do weird things. It might even destroy your images on saving!

AND: DOCUMENTS WITH VECTOR LAYERS SAVED IN KRITA 4.0 CANNOT BE EDITED IN KRITA 3.x!

You can have both Krita 3 and Krita 4 on the same system. They will use the same configuration (for now, that might change), which means that either Krita 3 or Krita 4 can get confused. They will use the same resources folder, so brush presets and so on are shared.

Downloads

Right now, all releases and builds, except for the Lime PPA, are created by the project maintainer, Boudewijn Rempt. This is not sustainable! Only for the Windows build, a third person is helping out by maintaining the scripts needed to build and package Krita. We really do need people to step up and help maintain the Linux and macOS/OSX builds. This means that:

  • The Linux AppImage is missing Python scripting and sound playback. It may be missing the QML-based touch docker. We haven’t managed to figure out how to add those features to the appimage! The appimage build script is also seriously outdated, and Boudewijn doesn’t have time to improve it, next to all the other things that need to be done and managed and, especially, coded. We need a platform maintainer for Linux!
  • The OSX/macOS DMG is missing Python scripting as well as PDF import and G’Mic integration. Boudewijn simply does not have the in-depth knowledge of OSX/macOS needed to figure out how to add that properly to the OSX/macOS build and packages. Development on OSX is picking up, thanks to Bernhard Liebl, but we need a platform maintainer for macOS/OSX!

Windows Download

Note for Windows users: if you encounter crashes, please follow these instructions to use the debug symbols so we can figure out where Krita crashes.

There are no 32 bits Windows builds yet. There is no installer.

Linux Download

(If, for some reason, Firefox thinks it needs to load this as text: to download, right-click on the link.)

OSX Download

Note: the gmic-qt and pdf plugins are not available on OSX.

Source code

md5sums

For all downloads:

Key

The Linux appimage and the source tarball are signed. You can retrieve the public key over https here:
0x58b9596c722ea3bd.asc
. The signatures are here.

Support Krita

Krita is a free and open source project. Please consider supporting the project with donations or by buying training videos or the artbook! With your support, we can keep the core team working on Krita full-time.

November 04, 2017

Things to Consume Your Time & Attention

A few things I’ve been enjoying recently:

The Vietnam WarThe Vietnam War by Ken Burns & Lynn Novick

The Vietnam War is a ten-part, 18-hour documentary from PBS. It’s available to stream for free on the PBS website/app, but only if you live in the US. For those of us fortunate enough not to live in the Greatest Country on Earth™, there are some hoops to jump through to watch it.

The easiest set of hoops I’ve found is to use the free built-in VPN in the Opera web browser. You can easily select the US as your VPN zone and enjoy the web as our freedom-loving friends see it.

The documentary is exhaustive and is propelled by remarkable interviews with participants from Both Sides™ of the war. It will leave you wondering how it could have ever been allowed to happen, and terrified that it will happen again.

How F*cked Up Is Your Management?How F*cked Up Is Your Management by Jonathan & Melissa Nightingale

Jonathan & Melissa Nightingale have gathered some of their best writing from their excellent blog, The Co-Pour. They discuss and share what they’ve learned about the mechanics and humanity of working with people in a business. It’s short, easy to read, has a profane title, and a fine domain name: hfuiym.com.

Hidden America with Jonah Ray

Hidden America with Jonah RayI came to know Johan Ray though his co-hosting of the Nerdist podcast. He plays the host of a fake travel show, Hidden America with Jonah Ray, which is generally a spoof of low-budget cable travel shows, but occasionally drifts into absurd horror.

Watching it from outside the US is frustrating. It’s a great show – but it’s barely worth jumping through the hoops required. Like The Vietnam War doc, you’ll need the Opera VPN to watch from outside of the US, then you need to create an account on the VRV site, and even then the show is peppered with ads that randomly interrupt playback. As I said, barely worth it – but still worth it.

If you’re not sure you want to jump through all of those hoops, the first episode (Boston) is on YouTube as is a short trailer.

Meet is Murder

I’d like to write a business-oriented self-help book, just so I could have a chapter titled “Meet is Murder.”

Now all I need is the contents for that chapter, and all of the other chapters. Keep an eye out for it on the best-seller lists in 8 to 10 years.

November 03, 2017

Krita 3.3.2 Released

Today we are releasing Krita 3.3.2, a bugfix release for Krita 3.3.0. This release fixes two important regressions:

  • Krita 3.3.1 would read brush presets with textures incorrectly. This is now fixed.
  • Windows 1709 broke wintab and Windows Ink tablet handling in various ways; we worked around that and it works again in this version of Krita.

Additionally, there are the following fixes and improvements:

  • Animation: make it possible to export empty frames after the end of the animation.
  • Animation: make it possible to render up to a 10,000 frames
  • Add a command-line option to start Krita with a new, empty image: krita --new-image RGBA,8,5000,3000
  • Performance: improved caching for effect and selection masks
  • Performance: Fix a leak in the smudge brush
  • Performance: Improve performance when using the hardware-accelerated canvas
  • Performance, Windows: improve the performance when loading icons
  • macOS: render the frames-per-second overlay widget correctly
  • Filters: it’s now possible to edit the filter’s settings directly in the xml that is used to save filter definitions to .krita files.
  • Filters: a new ASC_CDL color balance filter was added, with Slope, Offset and Power options.
  • Crashes: fix a crash that happened when closing a second document with infinite canvas active
  • Layers: Make it possible to copy group layers
  • UI: make it possible to use the scroll-wheel to scroll through patterns when the patterns palette is very narrow.
  • UI: Improve drag and drop feedback in the layer panel
  • UI: Hide the lock and collapse titlebar icons when a panel is floating
  • G’Mic: the included G’Mic is updated to the latest release.

Download

Windows

Note for Windows users: if you encounter crashes, please follow these instructions to use the debug symbols so we can figure out where Krita crashes.

Linux

(If, for some reason, Firefox thinks it needs to load this as text: to download, right-click on the link.)

When it is updated, you can also use the Krita Lime PPA to install Krita 3.3.2 on Ubuntu and derivatives. There is also an updated snap.

OSX

Note: the gmic-qt and pdf plugins are not available on OSX.

Source code

md5sums

For all downloads:

Key

The Linux appimage and the source tarball are signed. You can retrieve the public key over https here:
0x58b9596c722ea3bd.asc
. The signatures are here.

Support Krita

Krita is a free and open source project. Please consider supporting the project with donations or by buying training videos or the artbook! With your support, we can keep the core team working on Krita full-time.

Programming an ATtiny85, Part 1: Using C with a USBtinyISP

[ATtiny85 and USBtinyISP programmer] Arduinos are great for prototyping, but for a small, low-power, cheap and simple design, an ATtiny chip seems like just the ticket. For just a few dollars you can do most of what you could with an Arduino and use a lot of the same code, as long as you can make do with a little less memory and fewer pins.

I've been wanting to try them, and recently I ordered a few ATtiny85 chips. There are quite a few ways to program them. You can buy programmers specifically intended for an ATtiny, but I already had a USBtinyISP, a chip used to program Arduino bootloaders, so that's what I'll discuss here.

Wiring to the USBtinyISP

[ATtiny85 and USBtinyISP wiring] The best reference I found on wiring was Using USBTinyISP to program ATTiny45 and ATTiny85. That's pretty clear, but I made my own Fritzing diagram, with colors, so it'll be easy to reconstruct it next time I need it. The colors I used:

MISO yellow VCC red
SCK white MOSI green
RESET orange
or red/black
GND black

Programming the ATtiny in C

I found a couple of blink examples at electronut.in, Getting Started with ATtiny AVR programming, and in a Stack Exchange thread, How to program an AVR chip in Linux Here's some basic blink code:

#include io.h>
#include <utildelay.h>

int main (void)
{
    // Set Data Direction to output on port B, pins 2 and 3:
    DDRB = 0b00001000;
    while (1) {
        // set PB3 high
        PORTB = 0b00001000;
        _delay_ms(500);
        // set PB3 low
        PORTB = 0b00000000;
        _delay_ms(500);
    }

    return 1;
}
</utildelay.h>

Then you need a Makefile. I started with the one linked from the electronut page above. Modify it if you're using a programmer other than a USBtinyISP. make builds the program, and make install loads it to the ATtiny. And, incredibly, my light started blinking, the first time!

[ATtiny85 pinout] Encouraged, I added another LED to make sure I understood. The ATtiny85 has six pins you can use (the other two are power and ground). The pin numbers correspond to the bits in DDRB and PORTB: my LED was on PB3. I added another LED on PB2 and made it alternate with the first one:

    DDRB = 0b00001100;
[ ... ]
        // set PB3 high, PB2 low
        PORTB = 0b00001000;
        _delay_ms(500);
        // set PB3 low, PB2 high
        PORTB = 0b00000100;
        _delay_ms(500);

Timing Woes

But wait -- not everything was rosy. I was calling _delay_ms(500), but it was waiting a lot longer than half a second between flashes. What was wrong?

For some reason, a lot of ATtiny sample code on the web assumes the chip is running at 8MHz. The chip's internal oscillator is indeed 8MHz (though you can also run it with an external crystal at various speeds) -- but its default mode uses that oscillator in "divide by eight" mode, meaning its actual clock rate is 1MHz. But Makefiles you'll find on the web don't take that into account (maybe because they're all copied from the same original source). So, for instance, the Makefile I got from electronut has

CLOCK = 8000000
If I changed that to
CLOCK = 1000000
now my delays were proper milliseconds, as I'd specified. Here's my working attiny85 blink Makefile.

In case you're curious about clock rate, it's specified by what are called fuses, which sound permanent but aren't: they hold their values when the chip loses power, but you can set them over and over. You can read the current fuse settings like this:

avrdude -c usbtiny -p attiny85 -U lfuse:r:-:i -v
which should print something like this:
avrdude: safemode: hfuse reads as DF
avrdude: safemode: efuse reads as FF
avrdude: safemode: Fuses OK (E:FF, H:DF, L:62)

To figure out what that means, go to the Fuse calculator, scroll down to Current settings and enter the three values you got from avrdude (E, H and L correspond to Extended, High and Low). Then scroll up to Feature configuration to see what the fuse settings correspond to. In my case it was Int. RC Osc. 8 Mhz; Start-up time PWRDWN/RESET; 6CK/14CK+ 64ms; [CKSEL=1011 SUT=10]; default value and Divide clock by 8 internally; [CKDIV8=0] was checked.

More on ports and pins

There's more info on ATtiny ports in ATTiny Port Manipulation (Part 1): PinMode() and DigitalWrite()

Nobody seems to have written much about AVR/ATTINY programming in general. Symbols like PORTB and functions like _delay_ms() come from files in /usr/lib/avr/include/, at least on my Debian system. There's not much there, so if you want library functions to handle nontrivial hardware, you'll have to write them or find them somewhere else.

As for understanding pins, you're supposed to go to the datasheet and read it through, all 234 pages. Hint: for understanding basics of reading from and writing to ports, speed forward to section 10, I/O Ports. A short excerpt from that section:

Three I/O memory address locations are allocated for each port, one each for the Data Register - PORTx, Data Direction Register - DDRx, and the Port Input Pins - PINx. The Port Input Pins I/O location is read only, while the Data Register and the Data Direction Register are read/write. However, writing a logic one to a bit in the PINx Register, (comma sic) will result in a toggle in the corresponding Data Register. In addition, the Pull-up Disable - PUD bit in MCUCR disables the pull-up function for all pins in all ports when set.

There's also some interesting information there about built-in pull-up resistors and how to activate or deactivate them.

That's helpful, but here's the part I wish they'd said:

PORTB (along with DDRB and PINB) represents all six pins. (Why B? Is there a PORTA? Not as far as I can tell; at least, no PORTA is mentioned in the datasheet.) There are six output pins, corresponding to the six pins on the chip that are not power or ground. Set the bits in DDRB and PORTB to correspond to the pins you want to set. So if you want to use pins 0 through 3 for output, do this:

    DDRB = 0b00001111;

If you want to set logical pins 1 and 3 (corresponding to pins 6 and 2 on the chip) high, and the rest of the pins low, do this:

    PORTB = 0b00001010;

To read from pins, use PINB.

In addition to basic functionality, all the pins have specialized uses, like timers, SPI, ADC and even temperature measurement (see the diagram above). The datasheet goes into more detail about how to get into some of those specialized modes.

But a lot of those specialties are easier to deal with using libraries. And there are a lot more libraries available for the Arduino C++ environment than there are for a bare ATtiny using C. So the next step is to program the ATtiny using Arduino ... which deserves its own article.

November 02, 2017

FreeCAD Arch development news - October 2017

Hi there, Time for a new report on the development of href="https://www.freecadweb.org/wiki/index.php?title=Arch_Module">Architecture and BIM tools for href="http://www.freecadweb.org">FreeCAD. Remember, you can help me to spend more time working on this, by sponsoring me on href="https://www.patreon.com/yorikvanhavre">Patreon, href="https://liberapay.com/yorik">Librepay or directly (ask me for a PayPal email or bitcoin address). Campain and future development Since I just recently opened the Librepay...

Quirks in fwupd as key files

In my previous blog post I hinted at you just have to add one line to a data file to add support for new AVR32 microcontrollers and this blog entry should give a few more details.

A few minutes ago I merged a PR that moves the database of supported and quirked devices out of the C code and into runtime loaded files. When fwupd is installed in long-term support distros it’s very hard to backport new versions as new hardware is released. The idea with this functionalty is that the end user can drop an additional (or replace an existing) file in a .d directory with a simple format and the hardware will magically start working. This assumes no new quirks are required, as this would obviously need code changes, but allows us to get most existing devices working in an easy way without the user compiling anything.

The quirk files themselves are simple key files and are documented in the fwupd gtk-doc documentation.

November 01, 2017

FOSDEM 2018 – SDN/NFV DevRoom Call for Content

The SDN & NFV DevRoom is back this year for FOSDEM, and the call for content is open until November 16th. Submissions are welcome now!

Here’s the full announcement:

We are pleased to announce the Call for Participation in the FOSDEM 2018 Software Defined Networking and Network Functions Virtualization DevRoom!

Important dates:

  • Nov 16: Deadline for submissions
  • Dec 1: Speakers notified of acceptance
  • Dec 5: Schedule published

This year, as it has for the past two years, the DevRoom topics will cover two distinct fields:

  • Software Defined Networking (SDN), covering virtual switching, open source SDN controllers, virtual routing
  • Network Functions Virtualization (NFV), covering open source network functions, NFV management and orchestration tools, and topics related to the creation of an open source NFV platform

We are now inviting proposals for talks about Free/Libre/Open Source Software on the topics of SDN and NFV. This is an exciting and growing field, and FOSDEM gives an opportunity to reach a unique audience of very knowledgeable and highly technical free and open source software activists.

This year, the DevRoom will focus on the emergence of cloud native Virtual Network Functions, and the management and performance requirements of those applications, in addition to our traditional focus on high performance packet processing.

A representative, but not exhaustive, list of the projects and topics we would like to see on the schedule are:

  • Low-level networking and switching: IOvisor, eBPF, XDP, DPDK, fd.io, Open vSwitch, OpenDataplane, Free Range Routing, …
  • SDN controllers and overlay networking: OpenStack Neutron, Calico, OpenDaylight, ONOS, Plumgrid, OVN, OpenContrail, Midonet, …
  • NFV Management and Orchestration: ONAP, ManageIQ, Juju, OpenBaton, Tacker, OSM, network management, PNDA.io, …
  • NFV related features: Service Assurance, enforcement of Quality of Service, Service Function Chaining, fault management, dataplane acceleration, security, …

Talks should be aimed at a technical audience, but should not assume that attendees are already familiar with your project or how it solves a general problem. Talk proposals can be very specific solutions to a problem, or can be higher level project overviews for lesser known projects.

Please include the following information when submitting a proposal:

  • Your name
  • The title of your talk (please be descriptive, as titles will be listed with around 250 from other projects)
  • Short abstract of one or two paragraphs
  • Short bio (with photo)

The deadline for submissions is November 16th 2017. FOSDEM will be held on the weekend of February 3-4, 2018 and the SDN/NFV DevRoom will take place on Saturday, February 3, 2017. Please use the FOSDEM submission website to submit your proposals (you do not need to create a new Pentabarf account if you already have one from past years). You can also join the devroom’s mailing list, which is the official communication channel for the DevRoom.

The Networking DevRoom 2018 Organization Team

October 31, 2017

AVR32 devices in fwupd

Over 10 years ago the dfu-programmer project was forked into dfu-utils as the former didn’t actually work at all well with generic devices supporting vanilla 1.0 and 1.1 specification-compliant DFU. It was then adapted to also support the STM variant of DFU (standards FTW). One feature that dfu-programmer did have, which dfu-util never seemed to acquire was support for the AVR variant of DFU (very different from STM DFU, but doing basically the same things). This meant if you wanted to program AVR parts you had to use the long-obsolete tool rather than the slightly less-unmaintained newer tool.

Today I merged a PR in fwupd that adds support for flashing AVR32 devices from Atmel. These are the same chips found in some Arduino protoype boards, and are also the core of many thousands of professional devices like the Nitrokey device. You can already program this kind of hardware in Linux, using clunky commands like:

# dfu-programmer at32uc3a3256s erase
# dfu-programmer at32uc3a3256s flash --suppress-bootloader-mem foo.ihx
# dfu-programmer at32uc3a3256s launch

The crazy long chip identifier is specified manually for each command, as the bootloader VID/PID isn’t always unique for each chip type. For fwupd we need to be able to program hardware without any user input, and without any chance of the wrong chip identifier bricking the hardware. This is possible to do as the chip itself knows its own device ID, but for some reason Atmel wants to make it super difficult to autodetect the hardware by publishing a table of all the processor types they have produced. I’ll cover in a future blog post how we do this mapping in fwupd, but at least for hardware like the Nitrokey you can now use the little dfu-tool helper executable shipped in fwupd to do:

# dfu-tool write foo.ihx

Or, for normal people, you can soon just click the Update button in GNOME Software which uses the DFU plugin in fwupd to apply the update. It’s so easy, and safe.

If you manufacture an AVR32 device that uses the Atmel bootloader (not the Arduino one), and you’re interested in making fwupd work with your hardware it’s likely you just have to add one line to a data file. If your dfu-tool list already specifies a Chip ID along with can-download|can-upload then there’s no excuse at all as it should just work. There is a lot of hardware using the AT32UC3, so I’m hopeful spending the time on the AVR support means more vendors can join the LVFS project.

Recommended podcast: Stay Tuned with Preet

Recommended podcast: Stay Tuned with Preet. The podcast is hosted by former U.S. Attorney Preet Bharara. I’d start with episode 1: That Time President Trump Fired Me.

October 30, 2017

Interview with Erica Wagner

Could you tell us something about yourself?

I’m Erica Wagner, a STEAM Nerd, Teenpreneur, Author, Instructor, YouTuber and self-taught 2D and 3D artist. I’ve been doing graphic design for two years, 3D sculpting, voxel art, and 3d modeling for one year, and digital drawing for a little over six months. I’m a homeschool student. My mom uses the majority of my projects as a part of school.

Do you paint professionally, as a hobby artist, or both?

Currently I’m a hobby artist but learning different art forms so I can make my own games and eventually my own animations.

What genre(s) do you work in?

I work in mostly science, cyber, sci-fi, and nature. Most of the work I make is STEAM related due to loving those areas which include but is not limited to movies, shows, games and books. Movies such as Star Wars, Interstellar, and Guardians of The Galaxy. An example of the shows I have watched are Gravity Falls and Doctor Who. Some of the games I have played are Hack ‘n’ Slash, Portal 2, Niche, and Robocraft. Lastly, some of my favorite books are the Nancy Drew Series, and Jurassic Park 1 & 2.

Whose work inspires you most — who are your role models as an artist?

When it comes to 2D art it would be the following Twitter people: loishh, viiolaceus, Cyarine, and samsantala. I love their styles. Some of them have varying styles of cartoony, realistic, and some have a mix of both. The mixture of realistic and cartoon styles appeal to me because they are realistic in the proportions, details, and colors; yet also cartoony that you’d see in webisodes. I’m not sure what the correct name for this style is but I love it. I want to develop my own style that is similar to this realistic cartoony mix so I can make my own concepts, illustrations, designs, and textures for 3d models.

How and when did you get to try digital painting for the first time?

I’m not sure the exact date but it was sometime in late 2016. Even though I did download Krita and two other programs in 2015, I didn’t actually make anything with them until late 2016. I played and tested the brushes to see what they did. I finally made something for a challenge I created in October 2016 called Artober.

What makes you choose digital over traditional painting?

I have endless resources to use. Don’t get me wrong, I enjoy traditional drawing. I did it a lot when I was younger. I can’t imagine spending money to buy lots of pens, pencils, markers, and other things when at the time I was just doing it for fun. I’m more of a techy person, so doing it digitally lets me play with different brushes without wasting anything. Plus it’s easier to paint 3D models this way and it’s easier to make things for graphics for ads, thumbnails, merch designs, etc.

How did you find out about Krita?

In late 2015, I searched in Google “Free Alternatives to Paint Tool Sai”. At the time, I was downloading all kinds of programs and just playing around in them so see which ones I liked. A website called Alternitateto.net popped up with results of different programs to use instead of Paint Tool Sai. I tried three or four different programs, one of them being Krita.

What was your first impression?

I was so amazed at all the things I could do in Krita. I had all kinds of brushes for different things at the time I had no idea what for, I could make my own animations too! I knew I had no idea how to use these features to make my own stories and worlds come to life but that didn’t matter to me. The fact I had the resource to learn how to make my own designs, concepts, and illustration and an alternative to Photoshop and Paint Tool Sai was great for me. It was such a great program I wondered why I had never heard of it or seen it in tutorials on YouTube. I was really excited to have a program that had all the features I wanted and needed to start the learning process.

What do you love about Krita?

I love how versatile and powerful it is. I can make my own brushes, drawings, animations, vectors, and textures for 3D models. When you’re just starting to teach yourself digital drawing you don’t want to spend hundreds of dollars on programs like Photoshop or Paint Tool Sai, especially if you don’t know if you’ll actually make a career from digital drawing or even like it. With Krita, I feel I’m getting the same amount and powerful features as the big name artists with Photoshop or Paint Tool Sai. The possibilities are endless! I also love that I can customize the layout of Krita to work for me or what I’m doing.

What do you think needs improvement in Krita? Is there anything that really annoys you?

I would like to be able to open a project, my 3D model texture for example, and in the history see the brushes, textures, and patterns I used. Currently Krita only remembers what brushes you used when you last opened it and not the brushes, textures, and patterns you used in certain projects.

What sets Krita apart from the other tools that you use?

For me it’s the vector feature. I also do graphic design and since I’m learning other art forms to make my own props for my graphics this really helps me. When you do graphic design you can use raster images but it helps a lot if you have vector images. Vector images don’t lose quality when you size them up or down. Vector images are really useful when you make Merch designs, ads, thumbnails, cover art, and more. The vector feature is so easy to learn and use. Once I got my brother to use Krita, he used it to make shirt designs and remade his brand’s logo.

If you had to pick one favorite of all your work done in Krita so far, what would it be, and why?

My favorite is the texture for the 3d lowpoly model t-rex I made for a shirt design. This is my favorite because it was my first time painting a texture for a 3D model. Based on the program I was using, there were three ways to paint the model. I knew I wanted to get better at drawing so I decided to take my model’s UV map, which is basically the layout of a 3D object in a 2D cutout like form, and paint it in Krita. While following a tutorial, the model took 10 hours, the texture took 11 hours, and the last 8 hours were for last minute fixing of the model, texture, and making it ready to put on a shirt. Right now 3D is my strong suit so having the 2D texture I was happy with work correctly on the model after working on this whole project for a total of 29 hours just made my entire day. I was so proud of how it all turned out and it looked amazing on the shirt. I’m still new to digital drawing and lowpoly modeling so this was a great experience for me.

What techniques and brushes did you use in it?

I used the Krita ink gpen 25 and the smudge rake 2 brushes. I chose the colors of my brand ScienceHerWay which are white, black, neon and dark shades of pink, purple, and teal and then used some light and dark grey. Certain areas of the dinosaur I made darker to give some details such as the dark purple lines in the lips, a darker shade of the color used on the nails, elbow and knee joints, and a light shade of teal on the inside of the mouth for where he teeth would be. For the pink streaks on the dinosaur’s back and legs I made a line of neon pink with the ink gpen 25 brush and then used the smudge rake 2 brush randomly to make it look like a natural pattern until the neon pink line was gone. I repeated this process with the dark neon pink.

Where can people see more of your work?

http://www.ScienceHerWay.com
Twitter: https://twitter.com/ScienceHerWay
YouTube: https://www.youtube.com/ScienceHerWay
Sketchfab: https://sketchfab.com/VirtualVel
DeviantArt: https://virtualdesignervixen.deviantart.com/

Anything else you’d like to share?

I recommend trying art challenges and contests. It’s a great way for you to practice and get out of your comfort zone. Even try art collabs. As long as you find a supportive art community, you shouldn’t have to worry about your skill level when it comes to this. The point is to get to know other artists, practice, and have fun. At the time of writing this, I’m in an art collab myself. I’m still learning how to digitally draw while the others have been doing this for years. It may feel intimidating, but I’m collabing and meeting with people I’ve never meet before and we’re all having fun. Plus I can learn from them.

October 27, 2017

Giraffe, Tortoise? Girtoise!

Two Girtoises about to feast on cloud-rooted Bananeries on the plains of the seastern continent. These animals are also known as Toraffes or by their scientific name: Giradinoides. In German, they have the even better name Schiraffen. The Bananeries contain valuable vitamins and minerals which help the animals in maintaining smooth fur and strong shells.

Detail at full resolution:

Available printed on apparel, as poster and a few other forms.

Technical notes

This is a completely tablet-drawn work. With my trusty serial Wacom Intuos, still working as I keep compiling the module after every kernel update. Originally, I wanted to use Krita for the nice paintbrush engine and the canvas rotation. I found the later to be critical in achieving the smoothest curves, which is a lot easier in a horizontal direction. With what ended up being a 10000 x 10200 resolution and only 4 GiB RAM, I ran into performance problems. Where Krita failed, GIMP still worked, though I had to switch to the development version to have canvas rotation. At the end, GIMP’s PNG export failed due to it not being able to fork a process with no memory left! Flattening the few layers to save memory led to GIMP being killed. Luckily, there’s the package xcftools with xcf2png, so I could get my final PNGs via command line!


Filed under: Illustration, Planet Ubuntu Tagged: Apparel, GIMP, Krita, T-shirt, xcftools

The Google Weekend

Last week was the Google Summer of Code Mentors Summit, a yearly event organized by Google, where they invite mentors of the Google Summer of Code program, a program that pays students to work on open-source projects. This year, like last year, FreeCAD participated to GSOC. This year we had 4 really good students,...

October 26, 2017

Reading an IR Remote on a Raspberry Pi with LIRC

[IR remote with Raspberry Pi Zero W]

Our makerspace got some new Arduino kits that come with a bunch of fun parts I hadn't played with before, including an IR remote and receiver.

The kits are intended for Arduino and there are Arduino libraries to handle it, but I wanted to try it with a Raspberry Pi as well.

It turned out to be much trickier than I expected to read signals from the IR remote in Python on the Pi. There's plenty of discussion online, but most howtos are out of date and don't work, or else they assume you want to use your Pi as a media center and can't be adapted to more general purposes. So here's what I learned.

Update: this page is for Raspbian Jessie. If you've upgraded to Stretch, read the update, Reading an IR Remote on a Raspberry Pi Stretch with LIRC, then come back here and jump forward to Set up a lircd.conf.


Install LIRC and enable the drivers on the Pi

The LIRC package reads and decodes IR signals, so start there:

$ sudo apt-get install lirc python-lirc python3-lirc

Then you have to enable the lirc daemon. Assuming the sensor's pin is on the Pi's GPIO 18, edit /boot/config.txt as root, look for this line and uncomment it:

# Uncomment this to enable the lirc-rpi module
dtoverlay=lirc-rpi

Reboot. Then use a program called mode2 to make sure you can read from the remote at all, after first making sure the lirc daemon isn't running:

$ sudo service lirc stop
$ ps aux | grep lirc
$ mode2 -d /dev/lirc0

Press a few keys. If you see a lot of output, you're good. If not, check your wiring.

Set up a lircd.conf

You'll need to make an lircd.conf file mapping the codes the buttons send to symbols like KEY_PLAY. You can do that -- ina somewhat slow and painstaking process -- with irrecord.

First you'll need a list of valid key names. Get that with irrecord -l and you'll probably want to keep that window up so you can search or grep in it. Open another window and run:

$ irrecord -d /dev/lirc0 ~/lircd.conf

I had to repeat the command a couple of times; the first few times it couldn't read anything. But once it's running, then for each key on the remote, first, find the key name that most closely matches what you want the key to do (for instance, if the key is the power button, irrecord -l | grep -i power will suggest KEY_POWER and KEY_POWER2). Type or paste that key name into irrecord -d, then press the key. At the end of this, you should have a ~/lircd.conf.

Some guides say to copy that lircd.conf to /etc/lirc/ andI did, but I'm not sure it matters if you're going to be running your programs as you rather than root.

Then enable the lirc daemon that you stopped back when you were testing with mode2. In /etc/lirc/hardware.conf, START_LIRCMD is commented out, so uncomment it. Then edit /etc/lirc/hardware.conf as specified in alexba.in's "Setting Up LIRC on the RaspberryPi". Now you can start the daemon:

sudo service lirc start
and verify that it's running: ps aux | grep lirc.

Testing with irw

Now it's time to test your lircd.conf:

irw
Press buttons, and hopefully you'll see lines like
0000000000fd8877 01 KEY_2 /home/pi/lircd.conf
0000000000fd08f7 00 KEY_1 /home/pi/lircd.conf
0000000000fd906f 00 KEY_VOLUMEDOWN /home/pi/lircd.conf
0000000000fd906f 01 KEY_VOLUMEDOWN /home/pi/lircd.conf
0000000000fda05f 00 KEY_PLAYPAUSE /home/pi/lircd.conf

If they correspond to the buttons you pressed, your lircd.conf is working.

Reading Button Presses from Python

Now, most tutorials move on to generating a .lircrc file which sets up your machine to execute programs automatically when buttons are pressed, and then you can test with ircat. If you're setting up your Raspberry Pi as a media control center, that's probably what you want (see below for hints if that's your goal). But neither .ircrc nor ircat did anything useful for me, and executing programs is overkill if you just want to read keys from Python.

Python has modules for everything, right? The Raspbian repos have python-lirc, python-pylirc and python3-lirc, and pip has a couple of additional options. But none of the packages I tried actually worked. They all seem to be aimed at setting up media centers and wanted lircrc files without specifying what they need from those files. Even when I set up a .lircrc they didn't work. For instance, in python-lirc, lirc.nextcode() always returned an empty list, [].

I didn't want any of the "execute a program" crap that a .lircrc implies. All I wanted to do was read key symbols one after another -- basically what irw does. So I looked at the irw.c code to see what it did, and it's remarkably simple. It opens a socket and reads from it. So I tried implementing that in Python, and it worked fine: pyirw.py: Read LIRC button input from Python.

While initially debugging, I still saw those

0000000000fda05f 00 KEY_PLAYPAUSE /home/pi/lircd.conf
lines printed on the terminal, but after a reboot they went away, so they might have been an artifact of running irw.

If You Do Want a .lircrc ...

As I mentioned, you don't need a .lircrc just to read keys from the daemon. But if you do want a .lircrc because you're running some sort of media center, I did find two ways of generating one.

There's a bash script called lirc-config-tool floating around that can generate .lircrc files. It's supposed to be included in the lirc package, but for some reason Raspbian's lirc package omits it. You can find and download the bash script witha web search for lirc-config-tool source, and it works fine on Raspbian. It generates a bunch of .lircrc files that correspond to various possible uses of the remote: for instance, you'll get an mplayer.lircrc, a mythtv.lircrc, a vlc.lircrc and so on.

But all those lircrc files lirc-config-tool generates use only small subsets of the keys on my remote, and I wanted one that included everything. So I wrote a quickie script called gen-lircrc.py that takes your lircd.conf as input and generates a simple lircrc containing all the buttons represented there. I wrote it to run a program called "beep" because I was trying to determine if LIRC was doing anything in response to the lircrc (it wasn't); obviously, you should edit the generated .lircrc and change the prog = beep to call your target programs instead.

Once you have a .lircrc, I'm not sure how you get lircd to use it to call those programs. That's left as an exercise for the reader.

October 25, 2017

Jabra joins the LVFS

Some great news: the Jabra Speak devices are now supported using fwupd, and firmware files have just been uploaded to the LVFS.

You can now update the firmware just by clicking on a button in GNOME Software when using fwupd >= 1.0.0. Working with Jabra to add the required DFU quirks to fwupd and to get legal clearance to upload the firmware has been a pleasure. Their hardware is well designed and works really well in Linux (with the latest firmware), and they’ve been really helpful providing all the specifications we needed to get the firmware upgrade working reliably. We’ll hopefully be adding some different Jabra devices in the coming months to the LVFS too.

More vendor announcements coming soon too.