Biometrics 2
Face recognition
PV181 Laboratory of security and applied cryptography
Seminar 27. 11. 2019
Agáta Kružíková, 409782@mail.muni.cz
Martin Ukrop, mukrop@mail.muni.cz
Lecture structure
Seminar 1
1. Introduction
2. Fingerprints
3. Seminar activity
– Fake fingerprints
4. Homework
– Report on selected
biometric system
Seminar 2
1. Face recognition
2. Seminar activity
– Face biometric
SWOT analysis
3. Homework
– Age estimation
2
Real-life example
3
Face recognition – Input
• Single picture
• Video sequence
• 3D image
• Facial thermograms
4
Face recognition: The manual way
5
Face recognition: The automatic way
• Statistical
– Eigenface, PCA, LDA, ...
• Neural networks
– Microsoft: Face API
– Facebook: DeepFace
– VK: FindFace (“best results” in MegaFace comp.)
– Google: FaceNet
6
FindFace – example
Subway photo (left), social network photo (right)
7
Challenges in face recognition
• Illumination
• Pose
• Environment
– Noisy background
• Aging
• Feature occlusion
– Hats, glasses, hair, ...
• Image quality
– colour, resolution, ...
8
Testing sets (databases)
• Many databases:
http://www.face-rec.org/databases/
• Covering:
– Aging
– Ilumination
– Pose
– Expresion
9
OpenBR: Face recognition overview
10
Photo © 2016 openbiometrics.org
OpenBR face recognition – visualization
• Haar-cascade Detection
• Machine learning based approach where a cascade
function is trained from a lot of positive and negative
images.
• See video:
OpenCV Face
Detection: Visualized
https://vimeo.com/12774628
11
CV Dazzle: Anti face-detection
12
Photo © 2010-2016 Adam Harvey, CV Dazzle
CV Dazzle: Anti face-detection
13
Photo © 2010-2016 Adam Harvey, CV Dazzle
Microsoft: Face API
14
Automatic passport control
15
Biometric passports
• “Smart card”, contain NFC chip
• Two security levels:
- BAC: Reading your photo+personal information
(Try Android app Passport reader)
- EAC: Reading your biometrics
- Fingerprint, Face and Iris support
16
Face impersonation
17
Photo © 2016 Carnegie Mellon University, Accessorize to a Crime: Real and Stealthy Attacks on State-of-the-Art Face Recognition
Face impersonation
• Fooling deep-neural-networks-based face
recognition systems (e.g. Face++)
– Over 90% success rate
– The principle is more general
• "physically realizable and inconspicuous"
Sharif, Mahmood, et al. "Accessorize to a crime: Real and stealthy
attacks on state-of-the-art face recognition." Proceedings of the 2016
ACM SIGSAC Conference on Computer and Communications Security.
ACM, 2016.
18
KFC AliPay
- Introduced 2015
- Only one KFC in China
- See AliPay promo video at
https://www.theverge.com/2017/9/4/16251304/kfc
-china-alipay-ant-financial-smile-to-pay
19
Apple FaceID hacked
- Liveness detection feature
- In 2019 by researchers
- Hacked by usage of pair of modified glasses
- A victim has to sleep :-)
Source: https://threatpost.com/researchers-
bypass-apple-faceid-using-biometrics-achilles-heel/
147109/
20
Detecting sexual orientation from faces
21
Photo © 2017 Deep neural networks are more accurate than
humans at detecting sexual orientation from facial images.
Journal of Personality and Social Psychology
Detecting sexual orientation from faces
• Classifying sexual orientation (straight vs. gay)
on men/women photos
– Human success: 61% / 54%
– Neural networks: 81% / 71%
– Neural networks (5 images): 91% / 83%
• May be a privacy issue!
Wang, Y., & Kosinski, M. (in press). Deep neural networks are more
accurate than humans at detecting sexual orientation from facial
images. Journal of Personality and Social Psychology, 2017.
22
Mugshots
23
Face recognition ban
- San Francisco
- “Threat to civil liberties”
- Ban for government agencies (city police and
sheriff)
- Federal agencies not affected
- Reason: privacy issue
- Less accurate at people of colour
- For the supplier: step back
- www.banfacialrecognition.com
Gregory Barber, San Francisco Bans Agency Use of Facial-Recognition Tech. 2019, Wired.
24
Code of Ethics (ACM)
1. Society and human well-being
2. No harm for participants & risk analysis
3. Honesty (transparency)
4. No plagiarism
5. Respect privacy
6. Confidentiality
7. High quality & standards (competence)
8. Professional review
9. Inform society
Advancing Computing as a Science & Profession, ACM Code of Ethics and Professional Conduct. Online [2019]:
acm.org/code-of-ethics
25
Fun with biometrics
• InterSoB task
– https://how-old.net/
– Try to appear
as old as possible
26
Photo © 2016 Dominika Krejčí, InterSoB
Detour: SWOT analysis
• A.k.a. “SWOT matrix”
• From 1960s
• Strategic planning
technique related to
business competition
or project planning
• Widely applicable
27
SWOT example: Passwords
Strengths
• Well understood
• Legacy
• Intuitive usage
• Possibility of high entropy
28
Weaknesses
• Often low entropy
• Infinite ways to implement
• Policy differences
• Sticky note syndrome
• Threats related to storage
Opportunities
• FIDO 2.0 system
• Integration of SMS/OPT and
Push-to-Approve
Threats
• Bad attack understanding
• Long tail of replacement
• Usability issues
• The dark web
Example inspired by the RSAC 2018 talk Passwords and fingerprints and faces – Oh my! Comparing old
and new authentication by Jackson Shaw
Seminar task
• Do a SWOT analysis for a use case on face
recognition biometrics, work in groups of three
• Use cases:
a. Face authentication
on border crossing (passports)
b. “Pay by a smile”
for Internet card payments
c. 3D face authentication
for accessing bank vaults
d. Thermal face scans
securing nuclear power plant
29
Homework
Exploring automatic age estimation
30
Homework: Overview
• Investigate what influences age estimation
– In https://how-old.net/ (neural-networks based)
– Adjust our pictures again
• Submit to IS MUNI a single ZIP file with
– Report (PDF),
see next slide
– Used adjusted images
• Deadline:
4. 12. 2018 8:00
31
Homework: Overview
Step 1: State the hypotheses.
E.g., Wrinkles around the tails of eyes increase the estimated age.
Step 2: Set the criteria for a decision.
Set baseline (no wrinkles) and repeat measurement for different
wrinkles around tails of eyes.
Step 3: Compute the test statistic (if you know how).
In our simplistic case, take a look on measurements.
This is not necessary, if you don’t understand statistics well.
Step 4: Interpret the results.
The hypothesis should not be regarded as true based on these
data.
32
Homework: Good methodology
33
Measurements:
Martin 1 - 27
Martin 2 - 27
Martin 3 - 27
Martin 4 - 27
Martin 5 - 27
Homework: Report
• Write a summarizing report
– Your hypotheses and how you tested them
– Test at least 5 distinct features
• Concentrate on:
– Having a formulated hypotheses for each feature
(e.g. smoother skin decreases estimated age)
– Having several images supporting/falsifying your idea
• Avoid:
– Many changes in the face at once
– Radical changes (deleting half the face)
– Overgeneralization
34
Homework: Methodology, scoring
• Up to 10 points awarded
– Scoring rubric available in the Information system
– The rubric can help you understand what is important
in the task!
• Have a look at old homework submissions with
good methodology in the Study Materials.
– Special thanks to Jan Kvapil and Rao Arvind
for providing them.
35
Homework: Bad methodology
(but at least funny)
36