Visual assessment of BIPV retrofit design proposals for selected historical buildings using the saliency map method

1.1The Swiss urban planning law and solar installations

The Swiss Raumplannungsgesetz RPG (urban planning law) has been published in order to prevent the urban sprawl and to fully utilize the existing urban areas before occupying new, untouched rural lands. Generating clean energy with large-scale solar parks by destroying existing rural landscapes is therefore definitely not encouraged. Instead, small-scale building-integrated photovoltaic (BIPV) installations on the existing building facades are preferred. However, BIPV installations need to follow certain restrictions.

According to the RPG article 18a that came into force on May 1st 2014, it is stated that “Solaranlagen [...] dürfen solche Denkmäler (Kultur- und Naturdenkmälern) nicht wesentlich beeinträchtigen” (Solar installations [...] should not significantly influence the original appearance of the cultural and natural heritages) (RPG, 2014). ‘Cultural heritage’ hereby refers to historical buildings that are selected by the individual Swiss Cantons (Swiss Federal Constitution, 2014) and (to be) put into the cantonal inventory list of historical buildings. RPG article 18a was proposed with the purpose to ensure both a careful integration of new clean energy and protection of the heritage of Swiss urban and nature landscape. Regarding the historical buildings, aside from the RPG article 18a, the government has also published a document on their website, giving suggestions on the sensitive issues that a renovation work on a historical building may face (Bundesamt für Energie & Eidgenössische Kommission für Denkmalpflege, 2009). The general attitude of this document is to encourage keeping the historical buildings as original as possible, and the change after the solar facade renovation should be as less as possible compared to the buildings’ original state.

However, the exact meaning of the expression ‘not significantly’ in the RPG article 18a has been left to each cantonal/local authority to define. In this case, local authorities can refer to the canton government, commune (Gemeinde) or the local cultural heritage preservation department (Denkmalschutz). Some cantons have published guidelines on how to install solar panels on the house roof/facade so as to smoothen the process of solar installation applications (Kanton Basel, 2013, 2012; Kanton Luzern, 2014; Kanton Thurgau, 2009). In the guideline brochures, suggestions are being made so that the solar panels will ‘not significantly’ affect the existing historical building.

1.2Problem statement

The solar installation guidelines for historical buildings from the Swiss cantons are putting a lot of weight on its appearance rather than on its construction, and guidelines are mostly interpreted with semantic and qualified descriptors. The direct result of RPG Article 18a and its derivatives is that people of different backgrounds have different understandings on how solar panels are ‘not significantly’ affecting the historical buildings and whether they co-exist harmoniously or not. The biggest barrier for the solar installation often comes from local authorities with their disagreement on the changed appearance on the original building facade, leading to confusion and conflict between them and the applicants (Von Arx, 2013; Häne, 2012; Ludin, 2013; Stadt Zürich Hochbaudepartement, 2008; Vitelli, 2013; Zurbriggen, 2009).

The main reason for the conflict is because the applicants (building owner, architect and solar engineers etc.) and local authorities (e.g. cultural heritage preservation department) often have different interests in this matter. The building owners and the engineers prefer cost- and energy-efficiency of the solar installation, and attenuate its visual aspect subconsciously. Architects may weigh cost and energy-efficiency equally with the visual aspect of the building, while the cultural heritage preservation department focuses entirely on the visual aspect and harmony of the solar installation and the existing building (see Fig. 1).

Here people deal with the co-existence of qualified (visual) and quantified (cost, energy) descriptors. Qualified descriptors hereby refer to linguistic descriptions such as ‘good’, ‘ugly’, ‘not harmonious’ etc. to describe how one feels about the visual aspect of the solar installation. Quantified descriptors refer to energy conversion, cost etc. that can be presented with numbers. The drawback of a qualified and relative descriptor is that different preferences, different cases and different persons will have different opinions about what is called that the solar installation is ‘not significantly’ affecting the existing historical building. It is relative, subjective and not always reasonable. Quantified descriptors, on the opposite, are absolute, objective and the scale do not vary according to the case, situation or person. The co-existence of two entirely different value systems can lead to conflicts and hence the failure of the project. In order to guarantee a reasonable judgment, a coherent value system with a fixed evaluation scale should be developed. One way to do this is to develop a transformation system that can translate the qualified, subjective descriptor into a quantified, objective one, so as to ensure that everything is objective and absolute. In this paper, a method to quantify the term ‘significance’ is introduced, so as to be able to quantify the word ‘not significantly’ subsequently.

1.3Current approaches

Even though this paper deals with the visual assessment of BIPV installation, literature on visual assessments of both photovoltaics and solar thermal installations is being reviewed so as to gain a holistic perspective in this matter. The words solar panel and solar installation therefore refer to both photovoltaics and solar thermal installations.

The architectural and visual values of solar panels as a new building element are being discussed widely. Mostly people discuss whether the solar panel is well integrated into its environment (the building facade) by starting from evaluating its material, colour, form and detail integration, which is already an indirect investigation of the term ‘not significantly influencing the appearance of cultural heritage’ in an intuitive way. Guidelines and recommendations are developed on how to evaluate solar panels in a holistic manner (Frontini et al., 2013; Munari Probst & Roecker, 2009, 2011, 2012). BIPV is being analysed by dividing their main characteristics into three main categories – functional, constructive and formal, where the functional category includes the basic building envelope function and solar energy collection ability, questioning whether the BIPV product fulfil its multifunctional purpose as a facade element. The constructive category cares about whether the BIPV are integrated well into the overall facade construction. The formal category is evaluating the solar panels on sub-categories such as shape & size flexibility, pattern, colour, joint/frame, dummy part availability. The assessment gives + for positive,±for mean and – for negative evaluation of the above mentioned aspects. The result for a solar panel product is produced by listing all the categories in a list with + and – symbols. The evaluation process is made manually and intuitively, and offers three grades (positive, mean and negative) for each aspect.

Another way to discuss the term ‘not significantly’ is by investigating the visibility of the solar panels, namely to assess whether the solar panels are visible from the streets or not. This method was developed by using simple 2D geometry (Dessi, 2013). Others also focus on the visibility but have used a different approach. By adapting the Gaussian probability function, the probability of noticing the solar panel within the human visual scene is being calculated from a pure mathematical point of view (Xu, 2014). Both approaches, however, are radically simplified methods and only take the visibility element into consideration, where the colour, reflectivity, material properties, details of the solar panel and the overall building environment are omitted.

In another research, daylight reflection issues of BIPV are being singled out to study their impacts on the visual appearance of the building. By using Radiance software (WardOthers, n.d.) and annual weather data (Wittkopf et al., 2014; Yang et al., 2013), one is able to quantify whether the BIPV installation is ‘significantly’ visible by the amount of daylight reflection it produces. This kind of research only covers one small part of the visual aspect of solar facade and hasn’t taken the attention ability of human eye into consideration.

Another method covers a more complex energy refurbishment assessment for historical buildings with solar panels system. With the goal of achieving better visual results, one should aim for co-planarity, respect the lines of the existing building, shape, grouping, accuracy and visibility while installing the solar panels on the building envelope, on a difficulty scale ranging from 1 to 3, with 1 meaning that the goal is easy to accomplish, and 3 meaning very difficult to accomplish (López & Frontini, 2014). In this case, the evaluation process is different for each architecture, because each historical building has its unique aesthetic difficulties when it comes to solar panel installation. Therefore the standard of this method varies from case to case, and the deciding process of the difficulty scale is also subjective.

A more sophisticated method sets up a holistic benchmark for ‘good’ BIPV integration (Yu et al., 2009). The overall BIPV evaluation is calculated by the multiplication of the separated evaluation results of electricity generation, building integration and BIPV combined benefits, then multiply the result with a weight that comes from the evaluation of the aesthetic evaluation. The weight is higher when the BIPV is more appropriately installed, and lower when poorly installed, so that the distinction can be made that even when different BIPVs fulfil the same basic function, but by achieving different aesthetic results, each of them can still have distinctive overall results. The aesthetic weight is given based on evaluating the BIPV on items such as ‘pleasantness’, ‘uniqueness’, ‘specialness’, ‘obviousness’ etc. Those terms are qualified descriptors that do not have explicit nor quantified explanations, and the judgment therefore is flexible and subjective. Other similar BIPV evaluations are also available (Lucchi et al., 2014; Vassiliades et al., 2014).

1.4Limitations and challenges

From the examples in Section 1.2 it can be inferred that even though a better energy efficiency is the ultimate goal, normally it is the appearance of the historical buildings that stands in the spotlight. Its importance even outranks the other aspects of the historical buildings.

Therefore, despite the fact that most of the literature above has not referred specifically to the term ‘not significantly’, the evaluations all have the same goal, which is to make the solar installation exist with the existing building in harmony, or in other words – ‘not significantly affecting the cultural heritage’ in its appearance.

It is apparent from the literature review that regarding the ‘significance’ of solar installation on the building facade, it is often tried to transform the grading of linguistic descriptors into numeric ones. Usually set 1 with qualified terms is given (such as ‘bad’, ‘medium’, ‘good’), and each one of them is mapped into set 2 with quantified numbers (e.g. bad = 0, medium = 1, good = 2 and similar). There are two disadvantages with this kind of transformation method. One is that each person has his own opinion about the qualified terms. That is to say that every individual has his own subjective, individual and flexible opinion about what is called ‘bad’, ‘medium’ or ‘good’. The other drawback is that a reasonable, unified mapping process is lacking. Each author can decide the composition of set 1, set 2 and the mapping method based on his need, without having any theoretic reference for the mapping method, making it impossible to objectively compare the visual aspect of solar panel installations of different projects with a unified parameter.

Therefore considering these two disadvantages mentioned above, two things need to be investigated:

2.1Saliency

The English Oxford Dictionary defines the adjective salient as ‘most noticeable or important’. Saliency refers to the noticeability of an object (pixel, person etc.) within its visual environment. The saliency map is therefore a 3D map indicating the probability of obviousness and noticeability of each point within the human vision field.

2.2Saliency maps

Humans have the ability to rapidly scan through a certain visual field (25 to 50 ms per item) and recognize an object that is standing out from their environment, which relies solely on the basic instinct of human nature and is called the ‘bottom-up’ processing of human attention. This attention mode is developed during evolution so as to detect the potential danger in the surrounding environment. The second attention mode is the search for objects that are of interest to the person. With pre-selection criteria that interest him, it is called the ‘top-down’ processing. In this mode, the human can be in charge of his attention selection criteria. The speed of this visual scanning is much slower – 200 ms or more per item (Itti & Koch, 2001).

The concept of saliency map was introduced by Koch and Ullman (Koch & Ullman, 1985), where they assume that the higher the conspicuity (clear visibility) of a location in the visual scene, the higher the firing of the neuron in the eyes. This is based on the ‘bottom-up’ theory. Later, they have refined their approach where they distinctly define the early visual features of ‘bottom-up’ attention to be the intensity, the colour and the orientation of the input image (see Fig. 2) (Itti, Koch, & Niebur, 1998). Each of these features then will undergo a process called the ‘centre-surround’ operation, which is an imitation of the way the retinal ganglion cells’ receptive field (a part of the human eye) works – each receptive field is divided into the ‘centre’ area, and a concentric ring, the ‘surround’ area. If the light intensity on those two areas is different, then it will cause the eye to respond. E.g. a single light beam might increase response of the cell (difference in centre and surround areas), while an overall brightness in the scene (less difference between light intensities in centre and surround areas) would weaken the response (Hubel, 1995).

In this way contrast and hence edges of the objects are calculated. After the normalization processes, so as to globally promote maps that have small values but strong peaks, and discourage maps that have overall high values, the feature maps will be combined into three conspicuity maps. Those will be again linearly combined into a final saliency map. On the scale from 0 to 1, the higher the value of a spot, the more likely the human eye is noticing this spot (see Fig. 3).

They have also proposed a Winner Take All (WTA) mechanism. The ‘bottom-up’ saliency map is generated by iterative calculations of the colour, intensity and orientations value. At the beginning of each iteration, the most salient spot from the last iteration will be left out. This mechanism inhibits the saliency modelling returning to the same conspicuous place in the visual scene repeatedly. Instead, it allows the modelling to move to the next salient place in the next round of calculation.

The Itti-Koch saliency map is based on the theory of bottom up processing of human attention. Many other kinds of models are being developed after that, with Itti-Koch’s principle as the basis (Borji & Itti, 2013). A benchmark developed by Judd et al. (Bylinskii, Judd, Durand, Oliva, & Torralba, n.d.; Judd, Durand, & Torralba, 2012) lists all the existing saliency methods to evaluate the accuracy of those models. While e.g. the Judd Saliency model has achieved a much better accuracy score than the original Itti-Koch method, the principle of the Judd method is adding other pre-selection models on top of the Itti-Koch model, enhancing the recognition ability to identify cars, persons, faces and horizon within the visual scene (Judd, Ehinger, Durand, & Torralba, n.d.).

2.3Current application

The original goal of modelling the attention within a given visual scene is to simulate the realistic neural network models and to explain human attentional behaviours. This research area has been very active over the past 25 years (Borji & Itti, 2013). Aside from predicting the most salient spots within the human visual scene, inspired application of these models have also appeared. For example the saliency maps can help save the rendering time in high-fidelity image rendering. High quality rendering will only happen to places in an image where human attention is attached to, while rendering the rest of the image in low quality. This can result in a reduction of calculation time (Chalmers & Debattista, 2009; Longhurst, Debattista, & Chalmers, 2006). Another application is using the saliency map to help blind people in environments that are unfamiliar to them. This is achieved by using the three basic features (colour, intensity and orientation) to detect the indoor signs, after that the information can be translated into vocal or other kinds of information that are accessible to visually impaired people (Wang & Tian, 2011). Humanoid robots can benefit from saliency map information. The robot can be guided by the saliency maps, and its attention is then focused on the most obvious locations (Ruesch et al., 2008). The field of application for the saliency map is very broad. However, in architecture, its benefit has not been fully realized yet, even though architects are constantly heavily involved in visual experiments in their design works.

3.1Method

This assessment methodology is based on creating a reference rendering (‘as is’) and design proposal renderings (‘new’). The pre-processing stage includes digitally recreating historical buildings itself and different designs with BIPV on the computer as 3D CAD model. Renderings, which are computer-generated images, are being made from the models. The main processing work is calculating the saliency values of each rendering image. The post-processing step is to compare the saliency values of the pictures of the historical building before (‘as is’) and after BIPV installation (‘new’). After that, one can then see whether the BIPV are ‘not significantly’ enough to exist harmoniously with the existing building, and how much they have changed the existing building regarding attention-aspect. Since the general goal of the historical building protection is to change as less as possible, therefore a good BIPV design on the facade should cause as less saliency change in the visual scene as possible.

The specific workflow can be divided into five major steps (see also Table 1).

A photo of the building’s original state is taken. The viewpoint of the photo is set where there is the highest probability of seeing the building. Later the saliency of the visual scene will be investigated from this viewpoint.

3D digital models of the building are created. The photorealistic image that is produced from the 3D model is called rendering. All the important elements of the building facade are included in the model, such as the proportion, detail, colour, obvious texture etc. In creating the model, the face normal of the model surfaces always face outwards, so as to enable the later rendering with Radiance. The Radiance program is an advanced lighting simulation tool (Ward & Others, n.d.), and suits the purpose of rendering a photo-realistic picture well. The viewpoint of the rendering ‘as is’ is identical as the one of the photo ‘as is’.

On the basis of the before-mentioned 3D model and viewpoint, building renderings with different designs (2D renderings ‘new’) of the BIPV installations are generated.

Since clear sky condition (CIE sky definition) will cause hard shadows and strong reflections in the renderings. They will have strong impact on the saliency maps. In order to avoid this situation, all the photos and renderings are made under overcast sky condition, which won’t result in hard shadows and exaggerated brightness. This enables the cross-comparison of saliency distribution between different designs (even solar installations on different buildings) under a unified condition that is independent of the daytime and location. After the rendering process, identical adjustments/optimization procedures are being applied to each rendering in order to make them more similar to the photo ‘as is’.

Itti-Koch saliency map (Itti et al., 1998) are produced from the renderings ‘as is’ and ‘new’ by using a script written by Harel, Koch, and Perona (2006). The thus generated saliency map from the rendering ‘as is’ acts as the standard saliency map; its saliency values act as the standard saliency values.

Delta saliency maps are calculated by the difference between the standard saliency value and the saliency values from different design renderings ‘new’ (see formula (1)).

After comparison, the design with the smallest change in saliency values equals the least visual change through the BIPV installation, therefore the building is most similar to its original state, this design is then the most appropriate design for this historical building.

3.2Application

3.3Analysis

The visual scene of the human eye is rather rectangular, with the horizontal axis being longer, instead of having a square form and therefore the rendering size follows this aspect ratio. The original rendering pictures are being cropped with the proportion of 2 in length and 1 in height (image size 1181 × 591) due to this reason (see Fig. 6). The investigation area is emphasized on the area where the BIPV are being installed. The 2:1 proportioned rendering images, suits the human vision better on the one hand, on the other hand fits well into the investigation focus of the research. Saliency maps will be generated based on these cropped images. After the processing, the saliency map will be automatically resized to 148 × 74 (equals 10952 pixels in total), which is defined in the processing script (Harel et al., 2006).

After processing the images with the script written by Harel, Koch, & Perona (2006), saliency maps can be generated for the original building rendering ‘as is’ and all eight design renderings ‘new’ (see Figs. 7-9). While Fig. 8 provides an overall impression of the saliency value distribution, Fig. 9 can provide a more refined reading of the specific saliency values. On each saliency map ‘new’, the saliency values are given on a scale from 0 to 1, with 0 being the lowest probability of drawing people’s attention and 1 being the highest probability. Intuitively, it can be read from Figs. 8 and 9 that the highest peaks in saliency values come from the silhouette of the house, with values being approximately 0.6-0.9. The second highest peak comes from the porch area of the house because of its heavy black awning, with values varying from 0.5-0.6. The third obvious peak is caused by the black BIPV panels (regardless of their layout). They draw the most of the eye’s attention to themselves and cause the biggest change in attention distribution throughout the visual field, sometimes even topping the saliency value peaks from the building’s silhouette itself. The differences between each saliency map ‘new’ of BIPV with ‘old rose’ front glazings are not so straightforward. The numbers above each saliency maps in Figs. 8 and 9 show the average saliency value throughout each map. Even though it is stated that the average saliency value all stay within the range of 0.28-0.38, implying that the overall saliency value have not changed much, they do not reveal information about the peak values in the maps.

Therefore the delta saliency value between saliency map ‘as is’ and saliency maps ‘new’ is calculated, which can be achieved by the Equation (1). On the scale from 0 to 1, 0 means that there is absolute no change in saliency values between the saliency map ‘as is’ and saliency maps ‘new’, and 1 stands for the highest possible difference between these two kinds of maps. This delta saliency maps are shown in Fig. 10. The results proved our former intuitive readings from Figs. 8-9 to be right, that the black panels cause the largest saliency differences. The rest of the readings are still somehow blurred out, with a few vague white spots visible.

To make the deviance clearer, the pixels in the delta saliency maps that have a change in saliency value of more than 0.1 are counted (white area). Delta saliency values below the threshold of 0.1 are omitted because they are considered as noise that is produced during calculation:

The results are shown in Figure 11. The first line above each delta value map shows the percentage of white area in each map. Designs with black BIPV panels have more than 1000 pixels that have a change in saliency value, causing tremendous change in saliency distribution. The large white areas in the left of the delta value maps of column 1 reveal the special characteristic of saliency calculations – the saliency value of each point is calculated in relation with its visual environment. A change of colour, intensity or orientation will eventually lead to a butterfly effect – the saliency values of other points will also change accordingly. So we can conclude that an obvious change in BIPV glazing colour can cause the human eye to invest slightly more attention in the west facade of the villa.

Regarding the BIPV with ‘old rose’ glazing, the design in row 4, column 2 proved to be the one with the least pixel number (100) with changed saliency value. This means compared to other designs, this design will only cause a minor change in saliency value before and after the BIPV installation. The other design with similar result is the design in row 3 column 2, with 114 pixels having a change in saliency value.

In order to present the result in a clearer manner, the percentages of the pixels that have not changed in the saliency value in the saliency maps ‘new’ are being calculated (black area):

This is shown in the second line above each delta saliency map in Figure 11. This kind of result representation suits the purpose of keeping the historical building as original as possible better. The design in row 4, column 2 shows that 99.09% of the pixels in the rendering have stayed the same. BIPV designs with ‘old rose’ glazings generally have a rate of unchanged saliency value of more than 95% , while black BIPV designs generally have only less than 82% of pixels staying the same in saliency values. The higher the percentage of the unchanged pixel, the better the design is for the preservation of this historical building. This result is objective and the experiment results are reproducible at any time.