Clairvoyance

To return to the home page click here.

To go to Coaching questions page click here.

Introduction and Description

Humans have been taking care of their hair since the Palaeolithic age, and nowadays hairstyles have become a staple in many different cultures and come in a variety of styles. In the western world, trends are usually established when celebrities appear in public with a new haircut, this often leads to a large number of people wanting a similar haircut. People rush to their hairdressers in demand for the exact same haircut not worried about the fact that it might look differently on them than it did on that celebrity. This can be due to various reasons like the difference in hair thickness, head shape or even lack of specific high-quality products and care needed to produce and maintain the haircuts. There is no real way to visualize the haircut on oneself apart from photoshopping images or using apps that give a poor representation. This is where Clairvoyance comes in.

There are multiple scenarios in which Clairvoyance can excel. Here are two of them.

Scenario 1

You walk into the hairdresser's with a haircut in mind, you explain it to your hairdresser and they seem to understand it. They start cutting and styling your hair only for you to realise at the end that this is not what you wanted. The haircut you received is not what you expected the haircut to look like on you, and that you've actually prefered another haircut. There is no way, however, to roll back to before you got the haircut and you are stuck with an unsatisfactory hair do. With this modern product, you will now be able to avoid all these unnecessary and annoying issues.

Scenario 2

You're a hairdresser and a customer walks into your store with an idea in mind. They present you that idea to the best of their ability, and you start cutting, styling and/or colouring their hair. When you present them with the final result they seem unsatisfied. This can be due to numerous reasons, you could simply have made a mistake, but most likely the haircut or style they requested is not what they expected it to be.

Description

The Clairvoyance preview imaging technology would allow for an easy bridge between a customer's expectation and a hairdresser's understanding! This can help them make a final decision on a haircut as well as concretise the idea customers have in mind. The product would allow for hairstyle and hair colour to be displayed and be easily customizable. No more miscommunications or disappointed customers.

Final Product

The final product will be an app. It will be able to create a 3D-model of a person’s head out of 3 pictures and adopt a 3D-haircut-model on it. It will recognize the age and sex of the person automatically and show proper haircut options. The user will be able to modify the hair length and colour manually. Furthermore he will have the opportunity to view different styling options for each cut. These options will include links to styling tutorials and an overview over the for the style needed products. After the choosing process, the customer will be able to share his haircut with friends on social media platforms, such as Facebook, Instagram, Twitter, Snapchat or Pinterest. If he wants, he can localize the closest hairdresser offering the requested haircut. He can take a look at pictures of previous results of this specific hairdresser and post a picture of his haircut after visiting him. In addition to that the customer can load his new haircut, used hair colours and hair regarding hair issues (such as thyroid or skin problems) in his personal profile on his phone. In that way his data will be protected on his own device and he is still able to show it to the next hairdresser.

An extended version of the app will be offered for hairdressers. It has all functions the private version has, but allows the hairdresser to sign up a profile for his shop. The location of his shop will be saved there. In addition to that he can upload sample pictures or videos of our predesigned haircuts on his customers. Users of the app will find his profile under each specific haircut and will be able to check him out before making an appointment. The hairdresser has also the option to build new haircuts and upload them with a tool we provide.

Existing Work

Research

Large Pose 3D Face Reconstruction from a Single Image via Direct Volumetric CNN Regression

State of the art AI technique for reconstructing facial meshes from a single picture. Does so by using a novel neural network technique called volumetric CNNs. For Clairvoyance this is a large piece of the puzzle since it would let us create realistic renderings of person's face with a different hairstyle. It is not the whole puzzle though; see the section drawbacks below. An online interactive demo of this technique is available at http://cvl-demos.cs.nott.ac.uk/vrn/

Head Reconstruction from Internet Photos

Reconstructs a whole head from hundredths of images. Results of the paper do not appear good enough for production.

3D Face Hallucination from a Single Depth Frame

Reconstructs a whole head from a single depth camera image. Impressive results.

Existing products

(Maybe explain why our application is better than the applications already available. They seem more like toys, while our idea might actually be useful for both customers and hairdressers.)

3D Reconstruction Products

facegen

Creates a 3D model of a person's head from a single or multiple images. However, final results are not very personalized but instead are constructed from an existing database of the heads of many different people. The results seem more like avatars than realistic 3D models.

itseez3d

Reconstructs any object in 3D by recording 360 degrees around an object with an iPad. Requires the Structure Sensor third-party peripheral.

Challenges

See the existing work section first. We have found very little mention in literature or otherwise of automatically generating a 3D model of a person's whole head from a few images. Existing research seems not yet ready for production or requires a calibrated setup where the background is eliminated, lighting is constant and motion of the camera is controlled. There also exists work where relatively cheap depth cameras are used that provide impressive results, depth cameras are however not yet available on most mobile devices. However, there is promising work in reconstructing the face part of a head. It is possible to generalize the rest of a person's head. That is, "glue" the face part on top of an average head model. The drawback of this is that we lose important details (in the case of applying hairstyles) such as the shape of a person's ears.

User Survey

Two surveys were created in order to receive feedback about our product. One was created for regular people and the other for hairdressers because we expect both groups to be able to deliver valuable information, but also have differences of opinions. Professional application of this device may require different features and expectation than that of customers. The surveys were made using the SurveyMonkey.com site. This site allows for an easy distribution of the survey as well as a quick collection and analysis of the data.

The survey starts off with a normal life scenario without the product so people can relate to the problematic situation. This differs between hairdressers and their customers. The product is then described and explanations are given to show where the product's strengths are and how it can improve the user's life. After this explanation, a list of questions is given from which we aim to extract as much information about the feelings and expectations for the product.

The survey intended for the professional use of the product only differs in its description, in order help them relate more to the problem at hand.

Question 1

The first question helps us get a feel for the initial opinion of the user towards this product. This is very useful because it gives us a good idea of how people feel about a product after a first description and thus how likely it is that the users want the product. Five answers are then available which range from very negative to very positive to allow the user to express their emotions towards the product.

Question 2

Quality is an important factor for product development since it will influence the price the product will be retailed at. More importantly, it also gives us an idea of how reliable and dependent our product is.

Question 3

This question asks how innovative users think the product is. This gives us insight on whether our product is perceived as state of the art and how much it will impact the market of the targeted industry, in this case, the hair industry.

Question 4

Question 4 asks how necessary the product seems to the users. Higher necessity will lead to a higher demand for the product. We expect beauty salon's and hairdressers to have a particularly high need for the product since it can help them distinguish their establishment from those without this product.

Question 5

This gives us an idea of how many people would buy this product and how enthusiastic users are for it. Furthermore, it complements the previous question on the necessity of the product.

Question 6

For this question, the survey changes between the user version and the professional version. In the professional version, we aim to see to what extent they would use this product over whatever method they are currently using. The user version asks how it will affect their habits if they have this product or the hairdresser does, using this information we can see how beneficial and profitable the product is for the professionals.

User version

Professional version

Question 7

With this question, we can find out which elements people find most important for our product. This can help us focus our attention towards specific attributes to improve our user's satisfaction as well as solve as many of their issues as possible. It's important to remember however that all these aspects, even if they are not very popular, will be important for the product. And that users may find some parts crucial and only realise it when they aren't optimised.

Question 8

This question gives an idea of the rate of diffusion of the product. In addition, it shows how enthusiastic the user is about the product and its innovations.

Question 9

Question 9 features an open question which allows the user to give opinions about the strong points of this product. It shows what they find most important in the product and how it can improve their lives.

Question 10

The final question in the questionnaire allows people to give feedback and suggest improvements to our technology. In this way, we can make sure to not miss out on valuable features that complement this technology as well be aware of the flaws in the product and its uses.

Market Value

We estimated the market value of the product based on 3 influences. The first one was how much a customer would pay for the app by himself, the second which amount of money could be reached by advertisements in the app. Last but not least we also took a look at how much money could be earned by selling personal data.

Comparable Apps

Most cosmetic simulation apps use a picture as basis and project make-up or haircut on this picture. The haircut is not well adjusted or looks artificial. In addition to that, it is only possible to see the haircut in the front of the face. For changing the hair colour those apps try to extract the hair from the picture and give it another colour. This method only works when the background and skin colour are very different to the customer's original hair colour. I cannot be applied to predesigned haircuts. Sometimes apps ask for the hair structure, but it does not have any consequences for predesigned haircuts. Our approach builds a 3D model of the head and applies the haircut on it. It fits the head and is turnable. Furthermore, hair colours and length can be changed for each model exactly. The model considers the structure of the customer's hair.

After choosing a haircut our app also gives ideas to style the hair including tutorials and needed products. Cosmetic apps are mostly free when the customer buys the app without special functions. Special functions can be for example trend information or a taller number of haircuts. We chose to offer the hairdresser version of the app for 10 Euro and the private customer version for free. Comparable apps have about 100,000 and 50 Million downloads. If we assume 10,000 downloads in first 6 months worldwide, 10% of the hairdressers. Furthermore, we expect 100,000 downloads after 2 years. That would be 10,000 Euro in the first 6 months and 100,000 Euro after 2 years.

Instagram

Instagram takes about 16 Euro for 1000 looks on a sponsored picture. Assuming 10,000 users in the first 6 months we could take for each advertisement 1600 Euro in the first half year and 16,000 after 2 years. Assuming a range of 20 different products (e.g. gel, hairspray, salt water spray, hairdryer) that would mean 32,000 Euro after 6 months and 320,000 Euro after 2 years.

http://adage.com/article/news/costs-ad-prices-tv-mobile-billboards/297928/

Data

The list of rights apps in the section has been long. Access to camera, media, location and WLAN has to be given to most styling apps. It is not said what this data is used for. Personal data could easily be sold via moving profiles, filming, audio data collection or analysing media on the advice. All of this is nothing we can accept responsibility for. The only thing we should evaluate is an anonymous statistics about most tried or shared hairstyles and sell it to fashion magazines or cosmetic companies. It would help them to identify new trends for the coming season and prepare a concept. For the development of our app, it could also lead to adjusting our haircut portfolio.

Laws and Ethics

Laws surrounding the issue

The idea of the mirror doesn’t violate any of the human rights since you voluntarily visit the hairdresser and can still decide to not have your hair cut after you dislike what you see. It is legal to put a price tag on the ‘’hair scanning’’ because it takes time and labour hours can legally be charged. The idea even really nicely complies with Article 27.

Everyone has the right freely to participate in the cultural life of the community, to enjoy the arts and to share in scientific advancement and its benefits.

As the concept of the mirror is very approachable for people from all classes, genders/cultures or religions, it is very easy to let everybody share in this scientific advancement and its benefits.

Privacy

Privacy is a fundamental human right recognized in the UN Declaration of Human Rights, the International Covenant on Civil and Political Rights and in many other international and regional treaties.

In the Netherlands, a business must follow the Dutch Data Protection Act (Wet bescherming persoonsgegevens, Wbp). This act outlines and emphasizes the amount of care needed when handling a customer's personal data. Without such acts, the data would be vulnerable to criminal activity such as theft. Transparency, with the user, about the usage of the data is crucial, they should be informed of where and how their data is being used. In some cases, the data must be reported to the Dutch Data Protection Authority, DPA (Autoriteit Persoonsgegevens). In addition, the customer should be made aware of your details such as your address and company names as well as making sure they are informed when you intend to share their data with other companies. It is furthermore, mandatory to include a privacy statement on your website if you have one.

A model that can be used when dealing with privacy factors in multimedia information sharing context is what we will refer to here as the Adams' privacy model. The aim of this model is to help multimedia communication systems to "determine which information users regard as private, from whom, and in which context". Rather than dealing with privacy as a black/white (or binary yes/no) variable, the model attempts to define "privacy boundaries" which if breached will reduce the effectiveness of the system being used. Furthermore, what defines these boundaries is, in fact, the privacy "perception" of the users. According to Adams' model, the User is anyone who has their data transmitted either directly or indirectly. Direct information may, for instance, be a person's consumption habits or medical records, while indirect information might be their image or voice. In fact, the user may not actually be actively using the system and may be totally unaware that their data is being transmitted. As shown in the figure below, Adams' model defines three main factors which interact with one another to create the users' overall perception of their privacy. These factors are defined as:

• Information Sensitivity: is the primary privacy factor which the other factors affect to determine the perceived sensitivity level. Information Sensitivity in a way defines the users' perceptions of the confidentiality of the information being transmitted. Also, users' judgment of the sensitivity levels of the information is not binary (private/not private), but multi-dimensional with varying degrees of sensitivity.

• Information Receiver: is the users' perception of the person who receives and/or manipulates their information. Although a range of factors would influence the users' assessment of the Information Receiver, the issue of trust seems to be the most important one in defining the users' perception of the Information Receiver.

• Information Usage: is the users' perception of how and for what purpose their information will be used at present as well as in the future. The potential importance that the users attribute to the perceived Information Usage is crucial to the users' estimation of privacy risk/benefit trade-offs.

A model that can be used when dealing with privacy factors in multimedia information sharing context is what we will refer to here as the Adams' privacy model. The aim of this model is to help multimedia communication systems to "determine which information users regard as private, from whom, and in which context". Rather than dealing with privacy as a black/white (or binary yes/no) variable, the model attempts to define "privacy boundaries" which if breached will reduce the effectiveness of the system being used. Furthermore, what defines these boundaries is, in fact, the privacy "perception" of the users. According to Adams´ model, the User is anyone who has their data transmitted either directly or indirectly. Direct information may, for instance, be a person's consumption habits or medical records, while indirect information might be their image or voice. In fact, the user may not actually be actively using the system and may be totally unaware that their data is being transmitted. As shown in the figure below, Adams´ model defines three main factors which interact with one another to create the users´ overall perception of their privacy. These factors are defined as:

• Information Sensitivity: is the primary privacy factor which the other factors affect to determine the perceived sensitivity level. Information Sensitivity in a way defines the users' perceptions of the confidentiality of the information being transmitted. Also, users' judgment of the sensitivity levels of the information is not binary (private/not private), but multi-dimensional with varying degrees of sensitivity.

• Information Receiver: is the users' perception of the person who receives and/or manipulates their information. Although a range of factors would influence the users' assessment of the Information Receiver, the issue of trust seems to be the most important one in defining the users´ perception of the Information Receiver.

• Information Usage: is the users' perception of how and for what purpose their information will be used at present as well as in the future. The potential importance that the users attribute to the perceived Information Usage is crucial to the users´ estimation of privacy risk/benefit trade-offs.

Marketing Strategy

Social Media Concept

We decided to focus our marketing on social media. The reasons are obvious: Advertisement there is effective and cheap. Furthermore, it approaches exactly our target audience. People tending to use “new media” are also more likely to download an app. Especially apps such as Snapchat, Instagram or Pinterest focussing on pictures and the looks are very likely to be used by our audience.

To create our marketing as effective as possible, we created a concept for our brand including some basic social media rules:

Target audience

It is important to make target audience specific advertisement. Not every platform addresses its audience in the same way. Facebook users are for example 50-60 years in average, whereas users of Instagram, Snapchat, Pinterest and Twitter are mainly between 15 and 35 years old. (http://www.pewinternet.org/fact-sheet/social-media/)

Follower base

The more follower of the brand has, the more likely it is to be seen by a lot of people. In order to get a follower, it is important to first build a solid base. This can be done by adding all friends and friends of friends.

After that, it is important to find so-called “multiplier”. Those are people attracting many others and can be used to advertise our brand. Influencers or bloggers are examples of multipliers. We can make offers to beauty bloggers to link their channel in the styling tutorials in our app in exchange for being named in their videos or being connected to their accounts. We can comment and like what they do, in exchange for them liking and commenting our product. In social media, it is all about interaction. The technique here would be linking influencer with a high reachability in our tutorials and comment and like at channels of the more unknown ones, who are still thankful for every follower.

Connection

The user should feel emotionally connected to the brand. It is important to face users on a personal level and make the brand more human. That can be reached by providing insights in the personal life of the people behind the app. For example, the social media account of the brand can be given to one person of the team and let him or her post every day. First about what they do for developing the product, but secondly about their daily life. Hairdresser and influencer should also get the opportunity to take over the account for a month.

In addition to that, it is eligible to put some user stories online. Users are given a voice. They feel part of a community, in which they are welcome and appreciated. The same community influencer they adore is part of. Those stories can be for instance their experiences with the app or previous hair styling faux pas. Users also can be asked about their opinion of new predefined haircuts or styles.

To deepen the connection between users and brand influencer can be used even more intense. They already have the connection to users. A haircut called after the influencer or an influencer’s face as example model are ways to connect the brand to the user unconsciously to his reputation. Furthermore, links of well-known beauty bloggers can be listed in our styling tips. In exchange for this advertisement, influencer might use our app for their videos or pictures.

Pictures

1. The profile picture in every app should be the logo of the brand.
2. The title picture on Facebook should be a pic of a group of people using the app together. Positions should be open and the user should feel as a part of the group. In this way, he feels connected and welcome.
3. The highest reachability on pictures will be reached if pictures touch their audience emotionally.

Nostalgic pictures arouse nice and warm feelings. People will feel connected to their childhood and to the brand the same moment. It will generate trust. We can post a picture of a girl getting her hair done by her mother. Historic hairstyles (from the 20th, 40th, 70th, 90th) would also work. To increase the effect we can use pictures of idols people had in their childhood. At this point, it is necessary to keep attention to the user’s age in each social media.

Being funny is also a way to attract people’s attention. It causes sympathy and good feelings, which immediately connect with the brand. It is also a good way to get connect the brand and the user. Laughing about the same things causes bondings. In our concrete situation, we could let users post pictures of their failed haircuts in the past. We could also take pictures of animals instead of people and project the haircut model on them.

Hashtags

Hashtags are a way to tag or find posts on social media. The widest scope is reachable with eleven hashtags. We need a scientific and personal hashtag for our product, in our case we choose #clairvoyance. In addition to that is is always good to stick with trendy hashtags. A weekly overview you can find at this link: http://hashtagify.me/popular

Requirements, Preferences and Constraints

Requirements

1. The application allows the user to take pictures and form a picture set.
2. A picture set consists of 4 pictures, capturing the front, back, left and right side of the user's head.
3. The application has to be able to create a 3D model of the user's head out of a picture set.
4. The application is able to extract the specific side of the user's head from each picture in the picture set.
5. The application has to be able to apply a 3D model of a haircut on the head of the 3D model of the person’s head.
6. The user should be able to manually choose and change the applied hairstyle.
7. The application must be able to show the results of applying a 3D model of a hairstyle on the head of the 3D model of the person’s head (3D view).
8. The user is able to pan the 3D view by swiping left and right; thereby allowing the user to see the whole of his generated head and applied hairstyle.

Preferences

1. The product should offer a well-received user experience (as determined by user tests).
2. The product should offer a tutorial in which the user is told how to use the product.
3. The application will initially contain at least 3 hairstyles per gender (male and female).
4. The application will allow changing the colour of the hairstyles.
5. The application will be able to download and use new hairstyles from a central server hosting our hairstyle files.
6. The hairdresser can pre-select which hairstyles he/she is able to cut/style.
7. The hairdresser can pre-select which hairstyles can actually be cut/styled for the current client.
8. The application will provide a realistic lighting solution for the 3D view.
9. The user is able to select different lighting.
10. The user is able to select light colour.
11. The user is able to select light position.
12. The user is able to select light intensity.

Constraints

1. The application must run in iOS 10 devices.
2. The application must run suitably (as determined by user tests) on a fifth-generation iPad or later iPads.
3. The application must run on Android 8 devices.
4. The application must run suitably (as determined by user tests) on a Samsung Galaxy Tab S3.
5. The application must run suitably (as determined by user tests) on an Asus ZenPad 3S 10.

Prototype

The prototype will consist of an Android app, showcasing our vision in terms of user experience and functionality. Its primary purpose is to gauge interest in the final product to potential buyers and users.

Requirements

The prototype will have:

• User interface to take 4 pictures of the user.
• A 3D view showing a 3D mesh of a head.
• At least 2 different hairstyles. The exact number will depend on the time required to create (or download and use) a single hairstyle (TBD).
• User interface to select different hairstyles.
• The 3D view shows the selected hairstyle on top of the user's head.
• The 3D view is rotatable so that user can see his whole head with the selected hairstyle.

The prototype will not have:

• The ability to convert from picture to 3D mesh. Instead, we will pre-generate a 3D mesh.
• The ability to dynamically add the hair to the 3D mesh. Instead, we will pre-generate this.
• A realistic lighting solution. Instead, we will use a simpler lightning solution.

Implementation

The app is created in Android Studio using the programming language Kotlin and Java. Functionality such as taking pictures, showing a 3D view and selecting hair styles and hair colors are successfully implemented. The user interface has been designed in accordance with Android's standards for user interface design (Material Design). Furthermore, the 3D view is purposefully designed to be similar to SnapChat's interface for editing photos. Designing an user interface to be similar to existing and familiar user interfaces helps the user quickly understand our app without being explicitly told what to do.

• 
• 
• 

Discussion

In this section, we will focus on the what we were not able to do during this project, as well as areas where increased research and experimentation is needed.

Technical features

Since what we have as our final product for this course is a prototype, we don't have a fully functioning product. This prototype doesn't provide some important technical features such as the following.

High-quality resolution of the hair:

The current resolution of the hair is simply what Maya has to offer and while this is useful for a prototype, customers will request more accurate and realistic hair in order to improve the immersion of the application.

Hair type:

Although trends and culture can cause hairstyles to be similar within communities, there are many hair types that need to be taken into account. From straight hair to afros, hair varies in curliness, in fact, many people choose to go from one hair type to another. The current application does not take this into account due to the number of hairstyles made in the limited amount of time. This is an important feature because people sometimes enjoy straightening or curling their hair for a change and therefore seeing the possible drastic result is crucial.

Hair length is another factor that should not be ignored, although people can't instantly grow their hair at the hairdresser, people let their hair grow different amounts based on preference and sometimes people decide to go for drastic changes.

The thickness and the amount of hair on are variables affected by genes, age, and even stress. Many hairstyles require a certain degree of these variables in order to hold or even just to look similar. This is obviously an essential part of the final product.

Accurate hair movement and physics when rotating the face:

Since hair is made up of hundreds of thousands of thin hair strands, its individual as well general position can be changed by just a small breeze. Because we live actively and constantly move our heads and bodies around, our hair is subjected to this movement and moves depending on its length and the force issued onto it. In order to improve the realism of the application hair movement can be added such that people can visualize how their hair would move around and relate more easily to the displayed 3D image.

Hair reflectivity from various light sources:

Although this is a minor adjustment, it might be interesting to add different settings in which the hair is illuminated by different light sources and light types.

Accurate real-time customizability such as:

• Hair length customization
• Hair type (thickness or curls)
• Hair colour

Hair customizability is a crucial feature in this application because its the primary component that facilitates the interaction between the hairdresser and their customer. Furthermore, it allows for more creative freedom as well as quick visualization of the expected final product. Although our prototype already fulfils some of these features, it does so in a limited fashion and thus doesn't represent the full potential of these features.

App developments

Furthermore, some developments could be made to the app itself in regards to:

Data transfer from one app to the other:

It might be interesting to a customer or hairdresser to be able to share haircuts they would like to have or have been customizing or working on. This would either be possible by sharing it on social media and messaging apps or through the app itself if both parties have the app. Not only would that boost the popularity of the app without advertising, it would also make people more likely to download the app and try it out themselves.

Server storage of data for personal use application:

The prototype does not have a way to store personal information but this would be part of the final product. In this way, users will have a history of their haircuts that can be shared with any hairdresser and possibly any of their friends. This, however, requires backend services as well as a thorough understanding of privacy which has been expanded on in this wiki.

UI improvements to make the app look better:

The current interface is simple and effective, but by no means is it optimal in terms of aesthetic and design. This can be achieved by hiring a good frontend designer or by relying on user feedback to improve it.

Ratings in terms of difficulty for the desired hairstyle to be realised:

If the haircut you desire is quite complex to get right it might be more effective to go to a higher end hairdresser. By adding a rating both the customer and the hairdresser are aware of the risk they are going to take. This is a good selling point for more experienced hairdressers and could boost the competitiveness of the hairdressing sector.

Social Network

Additionally, it would be interesting to create a social network for the users which allows them to

• Share haircuts they have created using the app
• Rate and comment on other haircuts
• Rate and comment on hairdressers
• Save haircuts to a favourites tab to see how it would look on them

Using a social network, we can create a community for the application which would popularise its use and enhance the content of the app itself through community submissions.