Paper reading session at Matsuo lab, the university of Tokyo.

1. Weblab@The University of Tokyo
Procedural Modeling Using Autoencoder Networks
Yumer, M. E., Asente, P., Mech, R., and Kara, L. B.
Proceedings of the ACM Symposium on User Interface Software and Technology (UIST), 2015.
Presenter: Shuhei Iitsuka

2. Weblab@The University of Tokyo
Introduction: Problem
Procedural modeling (PM) … Geometry modeling created by a set of parameters and algorithms.
PM provides rich representation, but it’s difficult to configure a large number of parameters.
→ More intuitive exploration method is required.
Example of PM: Tree modeling with L-system.
L-system, Wikipedia
Screenshot from a conventional PM interface.
[Figure 7 in paper]
!?!?
2

3. Weblab@The University of Tokyo
Introduction: Contribution
Proposed a dimensionality reduction method for PM using autoencoders.
An augmentation for continuous latent space generation (without using variational inference!) is also proposed.
Contributions:
● A method to create a lower-dimensional representations of PM using autoencoders.
● A PM design system interface using an explore-and-select interaction.
3
Screenshot from the PM design system interface.
[Figure 3 in paper]

4. Weblab@The University of Tokyo
Why this paper?
● Future extraction should be a key component for generating variations of web design.
● Autoencoders are attracting great attention for future extraction, so I was searching a good application of them
in the field of visual design.
● When I queried “visual design” and “autoencoder” on Google Scholar, this paper was the top hit.
4

5. Weblab@The University of Tokyo
Agenda
1. Related Works
2. Proposed Method
3. Evaluation and Results
4. Discussion & Conclusion
5

6. Weblab@The University of Tokyo
Related Works: Procedural Modeling
From when L-system is proposed, procedural modeling (PM) has many applications in digital design.
In order to migrate the burden of parameter adjustment, two approaches are proposed.
● Targeted design
● Exploratory system
6
Cities modeled by procedural modeling.
[Yoav 2001]

7. Weblab@The University of Tokyo
Related Works: Workaround for Parameter Adjustment
Targeted Design
Streamlines parameter adjustment by providing high-level
design grammar.
PROS: Users are free from adjusting PM parameters
directly
CONS: Less user control over the generated models.
7
Exploratory System
By providing helpful features for exploration, ease the
burden to adjust the parameters (e.g. show frequently
used models, devised parameter dialogue)
PROS: Users can start modeling from their liking initial
model.
CONS: Users need to back to the conventional PM
approach to tweak the parameters.
Design grammar for plant modeling.
[Lintermann 1999]
Visualization of goodness estimated by using crowdsourcing.
[Koyama 2014]

8. Weblab@The University of Tokyo
Related Works: Deep Neural Networks and Autoencoders
Deep neural networks model abstract characteristics of data using nonlinear transformations.
An autoencoder is a special form of a deep neural network which tries to reproduce the input layer at the output layer.
Autoencoders is superior to other dimensionality reduction methods [Hinton 2006].
→ Employ autoencoders for dimensionality reduction in this research.
8

9. Weblab@The University of Tokyo
Proposed Method: User Experience
9
Screenshot from the proposed exploration system interface.
[Figure 2 in paper]

10. Weblab@The University of Tokyo
Proposed Model: Sampling the Training Data
● Authors employ Categorization tree (C-tree) in order to sample data uniformly with respect to shapes rather
than parameters.
○ Create quartets of shaped based on topological structure.
○ Construct the hierarchical tree assembling the quartets.
● After the initial tree is constructed, conduct subsampling
As denoted with red rectangles in the figure.
● Iterates (1) adding new sample, (2) computing a new C-tree,
(3) performing subsampling until the criteria is satisfied.
10
Example of C-tree.
[Figure 4 in paper]
Example of topological quartet.
[Huang 2013]
Close
Far

11. Weblab@The University of Tokyo
Proposed Method: Neural Network Architecture
● 5-hidden-layer symmetric autoencoder.
● In addition to PM parameters, shape features are included in the input layer in order to ensure geometric continuity.
○ A silhouette-based histogram extracted from Light Field Descriptors is utilized.
● Training is done layer-by-layer followed by a fine tuning of the entire structure.
● Weighted reconstruction error on shape features is applied to the first hidden layer.
11
Autoencoder architecture.
[Figure 5 in paper]
Adding shape feature (f)
for geometric continuity.
[Figure 5 in paper]
Shape features
PM parameters
Example of Light Field Descriptors.
[Chen 2003]

12. Weblab@The University of Tokyo
Proposed Method: Denoising
● Denoising is applied only to shape features in the results of comparison.
● PM parameters have no hidden process which introduce noise because they fully control the shape.
On the other hand, shape features are sensitive to image-level processing details.
→ Denoising only works on shape features.
12
datasets

13. Weblab@The University of Tokyo
Evaluation and Results: Setup
● PM rulesets
○ Containers: 72 parameters
○ Trees: 100 parameters
13
Example from Containers.
[Figure 12 in paper]
Example from Trees.
[Figure 10 in paper]

14. Weblab@The University of Tokyo
Evaluation and Results: Interpolation
● The latent space dimensionality is fixed to 2 or 3 in order to provide intuitive and easy-to-understand interface.
14

15. Weblab@The University of Tokyo 15

16. Weblab@The University of Tokyo
Evaluation and Results: Shape Continuity in the Reduced Dimensional Space
A latent space extracted from Container ruleset. It seems shape features are contributing to the shape continuity.
16
PM parameters + shape features PM parameters only

17. Weblab@The University of Tokyo
Evaluation and Results: Shape Continuity in the Reduced Dimensional Space
A linear interpolation from Trees ruleset.
17
PM params only
PM params +
shape params

18. Weblab@The University of Tokyo
Evaluation and Results: Shape Continuity in the Reduced Dimensional Space
● Authors evaluate the continuity by introducing a cumulative shape dissimilarity measure, which sums up the
feature difference between neighbors along the axes of the latent space.
● Results show that the dissimilarity decreases in both rulesets compared to the baseline (PM parameters only).
18
S: samples generated in the latent space.
N_i: neighbors of sample i along the axes of
the latent space.
f_i: the feature vector of sample i.

19. Weblab@The University of Tokyo
Evaluation and Results: User Study
90 design non-expert users are asked to design one favorite container using system #1, then asked to replicate it using
system #2.
Group A: PM params only (#1) → PM params + shape params (#2)
Group B: PM params + shape params (#1) → PM params only (#2)
Users replicated the original container better when using PM params + shape params.
19
PM params only
PM params +
shape params
PM params only
PM params +
shape params

20. Weblab@The University of Tokyo
Evaluation and Results: User Study
● Users using the proposed system designed and replicated the container faster and also satisfied with the quality
of the replica.
● Users using the proposed system also gave positive feedback on its usability.
20

21. Weblab@The University of Tokyo
Discussion & Conclusion
Limitation of the proposed method
● Dimensionality reduction can result in making some shapes inaccessible any more which can be achieved by the
original parameters.
○ Trade-off between representational capacity and usability of the interface.
○ “Future work will require more studies in the design space siualization techniques.”
Conclusion
● Introduced autoencoder neural networks for design dimensionality reduction.
● Demonstrated the combination of shape features and PM parameters generates the latent space which is
primarily organized by shape similarity.
● Evaluated that the proposed system improves user experience and productivity.
21

22. Weblab@The University of Tokyo
References
[Hinton 2006] Geoffrey E Hinton and Ruslan R Salakhutdinov. 2006. Reducing the dimensionality of data with neural networks. Science 313,
5786 (2006), 504–507.
[Yoav 2001] Yoav IH Parish and Pascal Muller. 2001. Procedural modeling of cities. In ACM SIGGRAPH. ACM, 301–308.
[Lintermann 1999] Bernd Lintermann and Oliver Deussen. 1999. Interactive modeling of plants. Computer Graphics and Applications 19, 1
(1999), 56–65.
[Koyama 2014] Yuki Koyama, Daisuke Sakamoto, and Takeo Igarashi. 2014. Crowd-powered parameter analysis for visual design exploration. In
UIST. 65–74.
[Huang 2013] S. Huang, A. Shamir, C. Shen, H. Zhang, A. Sheffer, S. Hu, and D. Cohen-Or. 2013. Qualitative organization of collections of shapes
via quartet analysis. ACM Trans. Graph. 32, 4 (2013), 71.
[Chen 2003] D. Chen, X. Tian, Y. Shen, and M. Ouhyoung. 2003. On visual similarity based 3D model retrieval. In Computer graphics forum, Vol.
22. 223–232.
22