-
Notifications
You must be signed in to change notification settings - Fork 0
Expand file tree
/
Copy pathvideoFlexMM.html
More file actions
183 lines (161 loc) · 10.2 KB
/
videoFlexMM.html
File metadata and controls
183 lines (161 loc) · 10.2 KB
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
<!DOCTYPE html>
<html lang="en">
<head>
<meta charset="utf-8">
<meta http-equiv="X-UA-Compatible" content="IE=edge">
<meta name="viewport" content="width=device-width, initial-scale=1">
<meta name="description" content="">
<meta name="author" content="nfaraj, claunay" >
<link rel="icon" type="image/png" href="data/P5_mini_logo.png" />
<link rel="shortcut icon" href="data/P5_mini_logo.png" type="image/png" />
<title>Claire Launay</title>
<!-- Bootstrap Core CSS -->
<link href="vendor/bootstrap/css/bootstrap.min.css" rel="stylesheet">
<!-- Custom Fonts -->
<link href="vendor/font-awesome/css/font-awesome.min.css" rel="stylesheet" type="text/css">
<link href='https://fonts.googleapis.com/css?family=Open+Sans:300italic,400italic,600italic,700italic,800italic,400,300,600,700,800' rel='stylesheet' type='text/css'>
<link href='https://fonts.googleapis.com/css?family=Merriweather:400,300,300italic,400italic,700,700italic,900,900italic' rel='stylesheet' type='text/css'>
<!-- Plugin CSS -->
<link href="vendor/magnific-popup/magnific-popup.css" rel="stylesheet">
<!-- Theme CSS -->
<link href="css/creative.min.css" rel="stylesheet">
<!-- HTML5 Shim and Respond.js IE8 support of HTML5 elements and media queries -->
<!-- WARNING: Respond.js doesn't work if you view the page via file:// -->
<!--[if lt IE 9]>
<script src="https://oss.maxcdn.com/libs/html5shiv/3.7.0/html5shiv.js"></script>
<script src="https://oss.maxcdn.com/libs/respond.js/1.4.2/respond.min.js"></script>
<![endif]-->
</head>
<body id="page-top" class="bg-secondary">
<nav id="mainNav" class="navbar navbar-default navbar-fixed-top">
<div class="container-fluid">
<!-- Brand and toggle get grouped for better mobile display -->
<div class="navbar-header">
<button type="button" class="navbar-toggle collapsed" data-toggle="collapse" data-target="#bs-example-navbar-collapse-1">
<span class="sr-only">Toggle navigation</span> Menu <i class="fa fa-bars"></i>
</button>
</div>
<!-- Collect the nav links, forms, and other content for toggling -->
<div class="collapse navbar-collapse" id="bs-example-navbar-collapse-1">
<ul class="nav navbar-nav navbar-left">
<li>
<a href="index.html">Home</a>
</li>
<li>
<a href="index.html#research">Research</a>
</li>
<li>
<a href="index.html#teaching">Teaching</a>
</li>
<li>
<a href="index.html#contact">Contact</a>
</li>
</ul>
</div>
<!-- /.navbar-collapse -->
</div>
<!-- /.container-fluid -->
</nav>
<header>
<div class="header-content">
<div class="header-content-inner">
<h1 id="homeHeading">Claire Launay</h1>
<h3>Postdoctoral researcher - Albert Einstein College of Medicine</h3>
</div>
</div>
</header>
<!-- /.row -->
<section id="Unsupervised Video Segmentation Algorithms Based On Flexibly Regularized Mixture Models">
<div class="container">
<div class="row">
<div class="col-lg-12">
<p align="justify">
<h3>Unsupervised Video Segmentation Algorithms Based On Flexibly Regularized Mixture Models</h3>
Claire Launay, Jonathan Vacher, Ruben Coen-Cagli</p>
<h3>Abstract</h3>
<p align="justify">
We propose a family of probabilistic segmentation algorithms for videos that rely on a generative model capturing static and dynamic natural image statistics.
Our framework adopts flexibly regularized mixture models (FlexMM) (Vacher 2022), an efficient method to combine mixture distributions across different data sources.
FlexMMs of Student-t distributions successfully segment static natural images, through uncertainty-based information sharing between hidden layers of CNNs.
We further extend this approach to videos and exploit FlexMM to propagate segment labels across space and time.
We show that temporal propagation improves temporal consistency of segmentation, reproducing qualitatively a key aspect of human perceptual grouping.
Besides, Student-t distributions can capture statistics of optical flows of natural movies, which represent apparent motion in the video.
Integrating these motion cues in our temporal FlexMM further enhances the segmentation of each frame of natural movies.
Our probabilistic dynamic segmentation algorithms thus provide a new framework to study uncertainty in human dynamic perceptual segmentation.
</p>
<p align="justify">
A communication paper for the 2022 ICIP conference can be found <a href="data/Icip2022.pdf" target="_blank">here</a>.
</p>
<h3>Additional content </h3>
<p align="justify">
Here are four videos comparing the results of static and dynamic segmentation algorithms based on FlexMMs.
</p>
<p align="justify">
High temporal stability:
<br>
<table >
<thead>
<tr><td style='text-align:center'>Original image</td> <td style='text-align:center'>Ground truth</td> <!-- <td style='text-align:center'>Temp prop + OF segmentation</td>--> <td style='text-align:center'>Temp prop segmentation</td> <td style='text-align:center'>Indep SIL1 segmentation</td></tr>
</thead>
<tbody>
<tr>
<td><img src="data/videoFlexMM/bike-packing_img.gif" width="200" height=auto style="margin: 10px 10px 0px 0px;" alt="Original image" ></td>
<td><img src="data/videoFlexMM/bike-packing_gt.gif" width="200" height=auto style="margin: 10px 10px 0px 0px;" alt="Ground truth" ></td>
<!-- <td><img src="data/videoFlexMM/bike-packing_wof.gif" width="200" height=auto style="margin: 10px 10px 0px 0px;" alt "Temp prop + OF segmentation"> </td>-->
<td><img src="data/videoFlexMM/bike-packing_tmp.gif" width="200" height=auto style="margin: 10px 10px 0px 0px;" alt="Temporal propagation segmentation"> </td>
<td><img src="data/videoFlexMM/bike-packing_indep_SIL1.gif" width="200" height=auto style="margin: 10px 10px 0px 0px;" alt= "Indep SIL1 segmentation"> </td>
</tr>
<tr>
<td><img src="data/videoFlexMM/hike_img.gif" width="200" height=auto style="margin: 10px 10px 0px 0px;" alt="Original image" ></td>
<td><img src="data/videoFlexMM/hike_gt.gif" width="200" height=auto style="margin: 10px 10px 0px 0px;" alt="Ground truth" ></td>
<!--<td><img src="data/videoFlexMM/hike_wof.gif" width="200" height=auto style="margin: 10px 10px 0px 0px;" alt "Temp prop + OF segmentation"> </td>-->
<td><img src="data/videoFlexMM/hike_tmp.gif" width="200" height=auto style="margin: 10px 10px 0px 0px;" alt="Temporal propagation segmentation"> </td>
<td><img src="data/videoFlexMM/hike_indep_SIL1.gif" width="200" height=auto style="margin: 10px 10px 0px 0px;" alt= "Indep SIL1 segmentation"> </td>
</tr>
</tbody>
</table>
</p>
<p align="justify">
Low temporal stability:
<br>
<table >
<thead>
<tr><td style='text-align:center'>Original image</td> <td style='text-align:center'>Ground truth</td> <!--<td style='text-align:center'>Temp prop + OF segmentation</td> --><td style='text-align:center'>Temp prop segmentation</td> <td style='text-align:center'>Indep SIL1 segmentation</td></tr>
</thead>
<tbody>
<tr>
<td><img src="data/videoFlexMM/longboard_img.gif" width="200" height=auto style="margin: 10px 10px 0px 0px;" alt="Original image" ></td>
<td><img src="data/videoFlexMM/longboard_gt.gif" width="200" height=auto style="margin: 10px 10px 0px 0px;" alt="Ground truth" ></td>
<!--<td><img src="data/videoFlexMM/longboard_wof.gif" width="200" height=auto style="margin: 10px 10px 0px 0px;" alt "Temp prop + OF segmentation"> </td>-->
<td><img src="data/videoFlexMM/longboard_tmp.gif" width="200" height=auto style="margin: 10px 10px 0px 0px;" alt="Temporal propagation segmentation"> </td>
<td><img src="data/videoFlexMM/longboard_indep_SIL1.gif" width="200" height=auto style="margin: 10px 10px 0px 0px;" alt= "Indep SIL1 segmentation"> </td>
</tr>
<tr>
<td><img src="data/videoFlexMM/color-run_img.gif" width="200" height=auto style="margin: 10px 10px 0px 0px;" alt="Original image" ></td>
<td><img src="data/videoFlexMM/color-run_gt.gif" width="200" height=auto style="margin: 10px 10px 0px 0px;" alt="Ground truth" ></td>
<!--<td><img src="data/videoFlexMM/color-run_wof.gif" width="200" height=auto style="margin: 10px 10px 0px 0px;" alt "Temp prop + OF segmentation"> </td>-->
<td><img src="data/videoFlexMM/color-run_tmp.gif" width="200" height=auto style="margin: 10px 10px 0px 0px;" alt="Temporal propagation segmentation"> </td>
<td><img src="data/videoFlexMM/color-run_indep_SIL1.gif" width="200" height=auto style="margin: 10px 10px 0px 0px;" alt= "Indep SIL1 segmentation"> </td>
</tr>
</tbody>
</table>
</p>
<p align="justify">
If you have any question, feel free to send us an email, we'll be happy to answer them.
</p>
</div>
</div>
</div>
</section>
<!-- jQuery -->
<script src="vendor/jquery/jquery.min.js"></script>
<!-- Bootstrap Core JavaScript -->
<script src="vendor/bootstrap/js/bootstrap.min.js"></script>
<!-- Plugin JavaScript -->
<script src="http://cdnjs.cloudflare.com/ajax/libs/jquery-easing/1.3/jquery.easing.min.js"></script>
<script src="vendor/scrollreveal/scrollreveal.min.js"></script>
<script src="vendor/magnific-popup/jquery.magnific-popup.min.js"></script>
<!-- Theme JavaScript -->
<script src="js/creative.min.js"></script>
</body>
</html>