1
00:00:00,000 --> 00:00:03,917
Audrow Nash: How can we make it
easier to build robots? Good

2
00:00:03,917 --> 00:00:07,703
developer tooling is a great
start, and one of the biggest

3
00:00:07,703 --> 00:00:11,163
parts of tooling in my mind is
visualization. In this

4
00:00:11,163 --> 00:00:15,211
interview, I talk with Adrian
Macneil, who is a co founder and

5
00:00:15,211 --> 00:00:19,454
the CEO of Foxglove. If you're a
robotics developer, you probably

6
00:00:19,454 --> 00:00:23,110
know them and use their
visualizer. If not, you probably

7
00:00:23,110 --> 00:00:27,092
want to check it out. What you
may not know is Foxglove has a

8
00:00:27,092 --> 00:00:31,205
whole data infrastructure from
recording data to getting it off

9
00:00:31,205 --> 00:00:35,187
the robot and selecting which
data you want in the cloud. And

10
00:00:35,187 --> 00:00:39,039
of course, their specialty,
visualizing it, which works for

11
00:00:39,039 --> 00:00:43,021
live data and recorded data. In
this interview, we talk about

12
00:00:43,021 --> 00:00:46,415
all the parts of Foxglove and
how they fit together,

13
00:00:46,415 --> 00:00:50,398
opportunities for startups to
build robot tooling, Foxglove's

14
00:00:50,398 --> 00:00:54,445
2.0 release and their decision
to move away from open sourcing

15
00:00:54,445 --> 00:00:58,427
their visualizer, and Foxglove's
upcoming one day conference,

16
00:00:58,427 --> 00:01:02,148
Actuate. Actuate looks like a
blast . Its goal is to have

17
00:01:02,148 --> 00:01:05,674
conversations where robotics
companies can share their

18
00:01:05,674 --> 00:01:09,525
learnings kind of like what's
done at academic conferences,

19
00:01:09,525 --> 00:01:13,377
but with more of an industry
focus. If you do decide to go,

20
00:01:13,377 --> 00:01:17,490
you can use the code Audrow, A U
D R O W, to get 20 percent off

21
00:01:17,490 --> 00:01:21,733
It's not a referral, I don't get
anything if you sign up, but I'm

22
00:01:21,733 --> 00:01:25,911
flattered to have a promo code.
Anyway, I hope to see you there.

23
00:01:25,911 --> 00:01:28,000
With that, here's the interview.

24
00:01:28,000 --> 00:01:30,000
Audrow Nash: Hey Adrian, would
you introduce yourself?

25
00:01:30,000 --> 00:01:34,641
Adrian Macneil: Hi Audrow, my
name is Adrian Macneil, I'm the

26
00:01:34,641 --> 00:01:37,000
co founder and CEO at Foxglove.

27
00:01:37,000 --> 00:01:41,079
Audrow Nash: Awesome. would you
tell me about Foxglove and how

28
00:01:41,079 --> 00:01:42,000
it came to be?

29
00:01:42,000 --> 00:01:43,000
Adrian Macneil: Yeah,
absolutely.

30
00:01:43,000 --> 00:01:46,737
Adrian Macneil: So Foxglove we
think of as a visualization and

31
00:01:46,737 --> 00:01:50,234
observability platform for
robotics. We've been around for

32
00:01:50,234 --> 00:01:53,609
about three years now as a
company. Prior to Foxglove, I

33
00:01:53,609 --> 00:01:57,407
spent five years working in the
self driving industry at Cruise

34
00:01:57,407 --> 00:02:00,360
in San Francisco. Worked on a
lot of the internal

35
00:02:00,360 --> 00:02:04,038
infrastructure, internal tools,
developer tooling, simulation

36
00:02:04,038 --> 00:02:07,594
infrastructure, a lot of those
pieces at Cruise. And really

37
00:02:07,594 --> 00:02:11,392
came out of Cruise feeling like
there was this enormous, number

38
00:02:11,392 --> 00:02:15,008
one, there was this enormous
opportunity for automation, and

39
00:02:15,008 --> 00:02:18,565
autonomy and robotics in the
world, seeing if we can, put a

40
00:02:18,565 --> 00:02:22,001
car on the streets of San
Francisco. how, many other sort

41
00:02:22,001 --> 00:02:25,196
of tasks are done by humans
today that are dangerous,

42
00:02:25,196 --> 00:02:28,632
repetitive, difficult, that are
honestly much easier than

43
00:02:28,632 --> 00:02:32,429
driving around San Francisco. if
we have self driving cars, how

44
00:02:32,429 --> 00:02:36,166
come we don't have self driving
tractors in an empty field or,

45
00:02:36,166 --> 00:02:39,723
self driving forklifts in a
factory yet? so I came out with

46
00:02:39,723 --> 00:02:43,159
this sort of huge excitement
about what was possible with

47
00:02:43,159 --> 00:02:46,535
autonomy, but at the same time,
really feeling that, the

48
00:02:46,535 --> 00:02:49,609
industry was being held back by
a lack of tools and

49
00:02:49,609 --> 00:02:53,286
infrastructure and frameworks.
having seen kind of everything

50
00:02:53,286 --> 00:02:57,083
we had to build from scratch at
Cruise. So that was the genesis

51
00:02:57,083 --> 00:03:00,821
of Foxglove was how can we push
forward the robotics industry?

52
00:03:00,821 --> 00:03:04,076
How can we, build tools and
infrastructure that really

53
00:03:04,076 --> 00:03:07,873
accelerate the industry? and we
started with this, piece around

54
00:03:07,873 --> 00:03:11,731
visualization. We started with,
the Foxglove Visualizer, putting

55
00:03:11,731 --> 00:03:15,469
that out there, making it easy
for folks to bring in different

56
00:03:15,469 --> 00:03:19,327
types of data, for example, from
ROS. or very quickly, early on,

57
00:03:19,327 --> 00:03:23,004
we started adding support for
other frameworks and other data

58
00:03:23,004 --> 00:03:26,500
formats like protobuf and
things. and letting people bring

59
00:03:26,500 --> 00:03:30,056
this, multi modal data into a,
into kind of a visualization

60
00:03:30,056 --> 00:03:33,552
framework, that was really where
we started. and this is a

61
00:03:33,552 --> 00:03:37,410
problem that's pretty unique to
robotics and a lot of these kind

62
00:03:37,410 --> 00:03:40,424
of, ML workflows. If you think
back to Traditional

63
00:03:40,424 --> 00:03:44,101
observability, you think about
tools like Datadog or Grafana,

64
00:03:44,101 --> 00:03:47,839
they're really built around time
series data, metrics, lots of

65
00:03:47,839 --> 00:03:51,516
plots, but no ability to show
you 3D data, no ability to show

66
00:03:51,516 --> 00:03:55,313
maps, no ability to show point
clouds or anything like that. So

67
00:03:55,313 --> 00:03:58,629
we started with the visualizer
and then over time we've

68
00:03:58,629 --> 00:04:02,246
expanded into this broader what
we think of observability as

69
00:04:02,246 --> 00:04:05,742
everything from how you're
logging on, a robot, how you're

70
00:04:05,742 --> 00:04:09,660
offloading data from robots, how
you get it to the cloud, how you

71
00:04:09,660 --> 00:04:13,397
store and index it, and then how
you can visualize and analyze

72
00:04:13,397 --> 00:04:14,000
that data.

73
00:04:14,000 --> 00:04:17,112
Audrow Nash: Yeah. So you're
carving out a very nice vertical

74
00:04:17,112 --> 00:04:19,000
with this all around. All things
data

75
00:04:19,000 --> 00:04:22,661
Adrian Macneil: Basically, yeah,
we see the whole, data life

76
00:04:22,661 --> 00:04:26,444
cycle of, from your recording on
the robot. because these are,

77
00:04:26,444 --> 00:04:29,861
things that are just, they work
quite differently from a

78
00:04:29,861 --> 00:04:33,400
traditional Infrastructure
space, right? Like I, I came to

79
00:04:33,400 --> 00:04:37,183
Cruise, I wanna say nine years
ago or something at this point,

80
00:04:37,183 --> 00:04:40,478
from an infrastructure
background, not from a robotics

81
00:04:40,478 --> 00:04:43,712
background. And I saw how how
much robotics engineers

82
00:04:43,712 --> 00:04:47,190
struggled with a lot of these
things. But the traditional

83
00:04:47,190 --> 00:04:50,973
tools that had built up around
sort of web servers and the web

84
00:04:50,973 --> 00:04:54,451
over the past couple decades did
not immediately apply to

85
00:04:54,451 --> 00:04:55,000
robotics.

86
00:04:55,000 --> 00:04:58,349
Adrian Macneil: So you think
about some of these things like,

87
00:04:58,349 --> 00:05:01,697
what types of data are being
logged, I mentioned, right? With

88
00:05:01,697 --> 00:05:04,936
websites, you're usually, the
logs coming out of like a web

89
00:05:04,936 --> 00:05:08,120
server is typically, that's
pretty lightweight in terms of

90
00:05:08,120 --> 00:05:11,359
data, you're generating a lot of
text logs, and then you're

91
00:05:11,359 --> 00:05:14,378
generating maybe some time
series metrics, things like,

92
00:05:14,378 --> 00:05:17,342
your server response speed,
memory use, and CPU on the

93
00:05:17,342 --> 00:05:20,746
server, and things like that.
But it's like pretty lightweight

94
00:05:20,746 --> 00:05:23,984
telemetry in the scheme of
things. and then, it's, and then

95
00:05:23,984 --> 00:05:27,278
this is deployed in a server
room, so you've got this, we're

96
00:05:27,278 --> 00:05:30,462
deployed in a data center, we've
got, effectively infinite

97
00:05:30,462 --> 00:05:33,756
bandwidth, and we've got, very
lightweight kind of telemetry

98
00:05:33,756 --> 00:05:37,324
coming off web servers. And then
you go to robotics, and it's the

99
00:05:37,324 --> 00:05:40,288
complete opposite on both of
those, right? You've got,

100
00:05:40,288 --> 00:05:43,637
instead of, very lightweight
telemetry, you've got incredibly

101
00:05:43,637 --> 00:05:47,095
dense data, sensor data, video,
you've got point clouds, you've

102
00:05:47,095 --> 00:05:50,499
got, all of this kind of data.
It's very multimodal, it's very

103
00:05:50,499 --> 00:05:53,463
heavyweight, quite often. and
then. Almost universally

104
00:05:53,463 --> 00:05:55,000
terrible internet, or, some,

105
00:05:55,000 --> 00:05:56,000
Audrow Nash: Yeah.

106
00:05:56,000 --> 00:05:59,513
Adrian Macneil: not a data
center, let's put it that way,

107
00:05:59,513 --> 00:06:03,396
you've, there's, you're either,
you're in a factory, you've got

108
00:06:03,396 --> 00:06:07,341
Wi Fi, you've got shitty Wi Fi,
you've got, or even if you, even

109
00:06:07,341 --> 00:06:10,670
if it's a fixed position robot
and it's connected with

110
00:06:10,670 --> 00:06:13,998
Ethernet, it's still in a
factory. There's still maybe

111
00:06:13,998 --> 00:06:17,943
you're lucky if you've got a 100
megabit uplink or something. or

112
00:06:17,943 --> 00:06:21,579
we deploy it into a, field where
you've got maybe like some

113
00:06:21,579 --> 00:06:25,031
spotty 3G service. you're
getting into maybe Starlink if

114
00:06:25,031 --> 00:06:28,667
you're lucky, but you're not
going to be streaming multiple

115
00:06:28,667 --> 00:06:32,119
HD video feeds over that. So you
invert on both of these

116
00:06:32,119 --> 00:06:35,940
spectrums. because of that. You
can't just go and reach for an

117
00:06:35,940 --> 00:06:39,084
existing off the shelf
observability Something like

118
00:06:39,084 --> 00:06:42,597
Datadog is just completely non
applicable to the robotics

119
00:06:42,597 --> 00:06:46,604
space. And yeah, we just, we see
this as something where robotics

120
00:06:46,604 --> 00:06:50,302
is, the needs of, robotics,
engineers and robotics companies

121
00:06:50,302 --> 00:06:54,000
is unique. and as, a specialized
tool for this is important.

122
00:06:54,000 --> 00:06:58,181
Audrow Nash: Now I would love to
go, I would love to go into each

123
00:06:58,181 --> 00:07:01,398
of the parts. of the you
mentioned so you have the

124
00:07:01,398 --> 00:07:05,000
visualization logging transport
like tell me about these

125
00:07:05,000 --> 00:07:09,127
Adrian Macneil: Yeah,
absolutely. walking through an

126
00:07:09,127 --> 00:07:14,048
order of how we see the, data
journey, the data life cycle. on

127
00:07:14,048 --> 00:07:18,492
a robot. recording is one of
the, the first ones that we

128
00:07:18,492 --> 00:07:20,000
think about, right?

129
00:07:20,000 --> 00:07:23,834
Adrian Macneil: one of our open
source projects at Foxglover is

130
00:07:23,834 --> 00:07:27,060
called MCAP. MCAP is a
standardized kind of recording

131
00:07:27,060 --> 00:07:30,833
and logging format. if you think
about recording in a robotics

132
00:07:30,833 --> 00:07:34,606
context, probably the, sort of,
The thing that a lot of people

133
00:07:34,606 --> 00:07:38,075
would be familiar with is
ROSBAGS, and the ROSBAG sort of

134
00:07:38,075 --> 00:07:41,910
file format was designed along
with ROS1, probably 15 years ago

135
00:07:41,910 --> 00:07:45,744
or more at this point, and, but
it was one of the first formats

136
00:07:45,744 --> 00:07:49,517
that sort of natively supports
recording multimodal streams of

137
00:07:49,517 --> 00:07:53,047
data, into a single file, and we
think that this is a very

138
00:07:53,047 --> 00:07:56,760
important property, being able
to take all of these different

139
00:07:56,760 --> 00:08:00,289
streams of sensor data, putting
it into a single file, the

140
00:08:00,289 --> 00:08:03,819
alternative, and we've seen this
at companies that sort of

141
00:08:03,819 --> 00:08:07,532
created pre mcap or, didn't use
ROS early on. the alternative

142
00:08:07,532 --> 00:08:11,062
typically ends up being a junk
drawer approach to logging,

143
00:08:11,062 --> 00:08:14,896
right? So there's just a folder
on the robot. there's some CSV,

144
00:08:14,896 --> 00:08:18,426
one piece of the stack is
logging some CSV data. One piece

145
00:08:18,426 --> 00:08:22,138
of the stack's logging an MP4
file. One piece of the stack is

146
00:08:22,138 --> 00:08:25,912
dumping some random binary data
to disk. and then there's some

147
00:08:25,912 --> 00:08:29,624
YAML files thrown in for good
measure. so this is junk drawer

148
00:08:29,624 --> 00:08:33,397
approach to logging. Now, great,
now we get these logs off the

149
00:08:33,397 --> 00:08:37,110
robot. It's just like a folder
full of crap. We zip it up and

150
00:08:37,110 --> 00:08:40,761
who knows, how you, go and,
interact with that later. but we

151
00:08:40,761 --> 00:08:44,170
think that this, nice property
that ROS bags originally,

152
00:08:44,170 --> 00:08:48,065
Created as everything should go
through a single format, even if

153
00:08:48,065 --> 00:08:51,777
it's completely different types
of data, even if it's some of

154
00:08:51,777 --> 00:08:55,550
it's structured data, some of
it's unstructured, some of it's,

155
00:08:55,550 --> 00:08:59,202
multimodal video point clouds,
some of it's time series. you

156
00:08:59,202 --> 00:09:02,854
want to put this all through a
single long format, make sure

157
00:09:02,854 --> 00:09:06,201
everything is timestamped,
consistent. That was a great

158
00:09:06,201 --> 00:09:09,853
property of ROS bags. but we
saw, the, robotics ecosystem is

159
00:09:09,853 --> 00:09:13,504
broader than ROS. We saw a lot
of companies struggling with,

160
00:09:13,504 --> 00:09:17,460
with where they were not using a
ROS stack. And therefore some of

161
00:09:17,460 --> 00:09:21,112
them weirdly converted data to
ROS just for logging. Some of

162
00:09:21,112 --> 00:09:24,885
them did the junk drawer quite a
few companies we spoke to had

163
00:09:24,885 --> 00:09:28,658
just invented their own sort of
bespoke binary logging format.

164
00:09:28,658 --> 00:09:32,188
So we've just, we've created
something internally. We just

165
00:09:32,188 --> 00:09:35,901
dumped these, this binary data
to disk. good luck interacting

166
00:09:35,901 --> 00:09:39,552
with any other services, or,
good luck even reading your own

167
00:09:39,552 --> 00:09:43,082
files from a year ago, because
who knows how that, how the

168
00:09:43,082 --> 00:09:46,734
format evolves. There's no kind
of spec to it. so this was a

169
00:09:46,734 --> 00:09:50,446
problem that we set out to solve
quite early on, back in 2021

170
00:09:50,446 --> 00:09:54,159
with Foxglove. We just, we saw
this as, A key sort of problem

171
00:09:54,159 --> 00:09:58,054
in the industry that there were
no open standards outside of ROS

172
00:09:58,054 --> 00:10:01,401
Bag that people were using for
logging created the MCAP

173
00:10:01,401 --> 00:10:04,809
project, that actually got
accepted I think maybe a year

174
00:10:04,809 --> 00:10:08,400
ago or a year and a half ago it
got accepted as the default

175
00:10:08,400 --> 00:10:12,052
logging format in ROS2 which is
great. We're seeing a lot of

176
00:10:12,052 --> 00:10:15,764
adoption of MCAP within the ROS
community, but also, a lot of

177
00:10:15,764 --> 00:10:19,294
adoption of MCAP outside of
ROS2. There's a number of some

178
00:10:19,294 --> 00:10:22,824
of them I think on the MCAP
website. Logos that you'll see

179
00:10:22,824 --> 00:10:26,232
of big companies. I don't want
to name ones that I'm not

180
00:10:26,232 --> 00:10:30,005
allowed to. So I'll just have a
look at what's actually listed

181
00:10:30,005 --> 00:10:33,535
publicly on the website. got,
yeah, there's some ones here

182
00:10:33,535 --> 00:10:37,004
like Andoril, Dexterity, Coco,
Apex AI, Tangram, Wabi. So

183
00:10:37,004 --> 00:10:40,839
there's a few of these, really
large companies, both within and

184
00:10:40,839 --> 00:10:44,003
outside of the robotics
ecosystem that have, adopted

185
00:10:44,003 --> 00:10:47,716
MCAP. So that's, how we think
about the logging piece, right?

186
00:10:47,716 --> 00:10:51,489
And because the logging pieces
on your robot. that needs to be

187
00:10:51,489 --> 00:10:55,262
open source, that needs to be a
completely open standard, open

188
00:10:55,262 --> 00:10:59,097
framework. MCAP has libraries in
about six different languages.

189
00:10:59,097 --> 00:11:02,505
There's, C and Python and Rust
and JavaScript and Go and

190
00:11:02,505 --> 00:11:05,000
anything that people are using
on robots.

191
00:11:05,000 --> 00:11:06,000
Audrow Nash: does it

192
00:11:06,000 --> 00:11:07,000
Adrian Macneil: and

193
00:11:07,000 --> 00:11:10,173
Audrow Nash: like so how would
you describe at a high level how

194
00:11:10,173 --> 00:11:13,043
does mcap work and how does it
unify things and why is it

195
00:11:13,043 --> 00:11:14,000
better than rossbag

196
00:11:14,000 --> 00:11:17,778
Adrian Macneil: Yeah, you can
think of MCAP as a successor to

197
00:11:17,778 --> 00:11:21,370
ROS1bags. so ROS1bags, was a
file format where it can take

198
00:11:21,370 --> 00:11:25,272
basically all of the channels of
ROS data, and save them into a

199
00:11:25,272 --> 00:11:28,306
single file format. Every
message that goes in is

200
00:11:28,306 --> 00:11:32,208
timestamped. There's an index at
the end of the file in the ROS

201
00:11:32,208 --> 00:11:35,738
bag that sort of helps you if
you need to seek around and

202
00:11:35,738 --> 00:11:38,959
things within the file. But
there was a few sort of,

203
00:11:38,959 --> 00:11:42,241
Challenges. The ROS one bag
format was not bad. a few

204
00:11:42,241 --> 00:11:46,081
challenges that people had with
it. One is that it was tied to

205
00:11:46,081 --> 00:11:49,983
the ROS ecosystem. not just, in
two ways. One is that you could

206
00:11:49,983 --> 00:11:53,761
only log ROS data to it. So if
you had other data, you've got

207
00:11:53,761 --> 00:11:57,600
some protobuf data or some just
other sort of generic data you

208
00:11:57,600 --> 00:12:01,254
wanted to save. you were back to
the juncture approach, now

209
00:12:01,254 --> 00:12:04,599
you've got to save that
separately. another way it was

210
00:12:04,599 --> 00:12:08,439
to ROS ecosystem is that the,
ROS, whatever Bagpile, whatever,

211
00:12:08,439 --> 00:12:11,969
the libraries to interact with
it are also only available

212
00:12:11,969 --> 00:12:15,932
through the ROS repos. You can't
just like pip install a general

213
00:12:15,932 --> 00:12:19,710
purpose package that would let
you interact with it. And this

214
00:12:19,710 --> 00:12:23,550
becomes a problem in, And, ML
like further on in your robotics

215
00:12:23,550 --> 00:12:26,771
development, you've got ML
pipelines, you've got ETL

216
00:12:26,771 --> 00:12:30,487
pipelines. You've got, you want
to do post processing on the

217
00:12:30,487 --> 00:12:34,388
data. You don't necessarily want
to install like all of ROS, or

218
00:12:34,388 --> 00:12:38,228
set up the ROS repos or any of
this kind of stuff. You've just

219
00:12:38,228 --> 00:12:42,068
got a standard sort of, Python
app that you've written and you

220
00:12:42,068 --> 00:12:45,598
want to just pip install
something interact with it. That

221
00:12:45,598 --> 00:12:49,128
was not possible. in the past
with ROSBags. So you had, a

222
00:12:49,128 --> 00:12:53,092
couple of these challenges, and
then in the early days of ROS 2,

223
00:12:53,092 --> 00:12:56,870
there was a switch to, a SQLite
based, logging format, which,

224
00:12:56,870 --> 00:13:00,524
it's, unclear that sort of some
of the design decisions and

225
00:13:00,524 --> 00:13:04,302
people behind the history of how
that decision came to be has

226
00:13:04,302 --> 00:13:07,956
been lost, I think, to the,
sands of time. was one, it was,

227
00:13:07,956 --> 00:13:11,858
it appears to be a decision that
was made in the early days. of

228
00:13:11,858 --> 00:13:15,759
ROS, but, and I think the intent
behind the SQLite login format

229
00:13:15,759 --> 00:13:19,475
was to bring it as I mentioned,
ROSbags was hard to interact

230
00:13:19,475 --> 00:13:22,634
with outside of the ROS
environment. The intent was

231
00:13:22,634 --> 00:13:26,474
like, hey, let's use an existing
open standard SQLite. There's

232
00:13:26,474 --> 00:13:30,128
plenty of readers out there that
would let you unpack that.

233
00:13:30,128 --> 00:13:33,534
Unfortunately, the downside of
that is that SQLite is a

234
00:13:33,534 --> 00:13:37,436
relational database, so it's not
designed for, high performance

235
00:13:37,436 --> 00:13:41,337
stream, append only logging or
anything like this. It's not, in

236
00:13:41,337 --> 00:13:44,744
terms of how you think about
inserting a row into a SQL

237
00:13:44,744 --> 00:13:48,336
database, it's hyper optimized
for jumping around and, and

238
00:13:48,336 --> 00:13:51,928
inserting rows arbitrarily and
updating indexes and all of

239
00:13:51,928 --> 00:13:55,706
these things that, are not at
all necessary when you've got a

240
00:13:55,706 --> 00:13:59,731
stream of log data that you just
want to flush to disk as fast as

241
00:13:59,731 --> 00:14:03,447
possible. so people were seeing
a lot of problems with that.

242
00:14:03,447 --> 00:14:07,287
Notably, it would basically just
drop a lot of messages, if it

243
00:14:07,287 --> 00:14:11,065
ran out of CPU or, the CPU was
being too taxed, it would just

244
00:14:11,065 --> 00:14:14,843
start dropping messages, and
then they would not make to your

245
00:14:14,843 --> 00:14:18,249
log file, there were some
challenges being faced there,

246
00:14:18,249 --> 00:14:22,027
and then, yeah, with MCAP, the
design of the MCAP file format

247
00:14:22,027 --> 00:14:25,991
is quite similar to MCAP, A ROS
1 bag. The other analogy I would

248
00:14:25,991 --> 00:14:29,645
say with an MCAP file is it's
similar to an MP4 file. So an

249
00:14:29,645 --> 00:14:33,051
MP4 file, people think of as a
video file, but it's not

250
00:14:33,051 --> 00:14:36,643
actually. An MP4 is just a
container. You can put anything

251
00:14:36,643 --> 00:14:40,359
inside it, and inside the MP4
file you're putting streams of

252
00:14:40,359 --> 00:14:44,137
typically something like H. 264
encoded video, plus some, AAC

253
00:14:44,137 --> 00:14:47,915
encoded audio, or whatever. MP3
encoded audio or any of these

254
00:14:47,915 --> 00:14:51,755
kind of things, right? So it's a
container. MP4 is a container

255
00:14:51,755 --> 00:14:55,285
format Just like Matroska is
also a container format, MKV

256
00:14:55,285 --> 00:14:59,125
These are containers and then
inside them you have channels of

257
00:14:59,125 --> 00:15:03,088
audio and video MCAP is similar
to that except inside it you can

258
00:15:03,088 --> 00:15:06,495
have channels of anything. You
can have channels of ROS

259
00:15:06,495 --> 00:15:10,334
messages, you can have channels
of CDR messages from, ROS2 and

260
00:15:10,334 --> 00:15:14,174
DDS, you can have channels of
protobufs, you can have channels

261
00:15:14,174 --> 00:15:17,828
of flatbuffers, you can have
channels of JSON, whatever you

262
00:15:17,828 --> 00:15:21,668
like. So think about it as just
a container format, everything

263
00:15:21,668 --> 00:15:25,570
that goes in the container, is
timestamped so that it fits into

264
00:15:25,570 --> 00:15:29,410
the sort of stream that you're
writing. and yeah, then you can

265
00:15:29,410 --> 00:15:33,064
go and unpack that later in.
Like I said, the libraries are

266
00:15:33,064 --> 00:15:36,841
available kind of independent
from the ROS ecosystem. even if

267
00:15:36,841 --> 00:15:40,000
you're using it within ROS,
that's a nice property.

268
00:15:40,000 --> 00:15:44,095
Audrow Nash: Yeah. Okay. That
sounds really nice. So you have

269
00:15:44,095 --> 00:15:47,719
this container where you can
store all these different

270
00:15:47,719 --> 00:15:51,881
channels, you can pull them out
really nice. And then you also

271
00:15:51,881 --> 00:15:56,177
have different language support
that don't, that doesn't require

272
00:15:56,177 --> 00:16:00,339
you to use, or install ROS on
ecosystem, use Ubuntu or Windows

273
00:16:00,339 --> 00:16:04,031
or whatever. whatever it's
supporting, but it's a heavy

274
00:16:04,031 --> 00:16:08,126
dependency. And so that's, very
nice. and that, that probably

275
00:16:08,126 --> 00:16:12,422
makes it a lot nicer for, as you
were saying, like ML pipelines,

276
00:16:12,422 --> 00:16:16,046
people who are doing data
analysis, they don't need to

277
00:16:16,046 --> 00:16:19,000
maintain a working ROS
distribution on their

278
00:16:19,000 --> 00:16:22,049
Adrian Macneil: Yeah, even
within Foxglove platform, some

279
00:16:22,049 --> 00:16:25,313
of our backend services that
ingest MCAP files or stream MCAP

280
00:16:25,313 --> 00:16:28,202
files are written in Go. for
example, and they have no

281
00:16:28,202 --> 00:16:31,358
dependency on ROS. We're not
running on, on, Ubuntu 2004 or

282
00:16:31,358 --> 00:16:34,675
outdated version of Ubuntu tied
to a specific ROS distribution

283
00:16:34,675 --> 00:16:37,671
or anything. It's just, hey,
we've installed the mcap go

284
00:16:37,671 --> 00:16:40,934
package and that has the, has
everything you need to interact

285
00:16:40,934 --> 00:16:44,144
with it. So you have this sort
of like level of independence

286
00:16:44,144 --> 00:16:45,000
that I think is,

287
00:16:45,000 --> 00:16:47,000
Audrow Nash: Very desirable.

288
00:16:47,000 --> 00:16:49,830
Adrian Macneil: and then of
course you still have the, tight

289
00:16:49,830 --> 00:16:52,000
ROS integration for people that
are using ROS.

290
00:16:52,000 --> 00:16:56,118
Audrow Nash: Definitely. Okay.
and so that's the recording

291
00:16:56,118 --> 00:17:00,308
part. So you're putting it in a
format that you can use. So

292
00:17:00,308 --> 00:17:04,000
MCAP, does it refer to anything
or is it an acronym?

293
00:17:04,000 --> 00:17:06,957
Adrian Macneil: it's,
unofficially it's Message

294
00:17:06,957 --> 00:17:10,607
Capture, which is a play on
PCAP, which Packet Capture. we

295
00:17:10,607 --> 00:17:14,445
don't expand that anyway, we
call it MCAP, and, it was, yeah,

296
00:17:14,445 --> 00:17:18,094
we went through, had a sort of
internal voting on about 50

297
00:17:18,094 --> 00:17:21,492
different names, I think, but
MCAP, MCAP, I don't even

298
00:17:21,492 --> 00:17:25,016
remember what the other
suggestions were, this was three

299
00:17:25,016 --> 00:17:28,728
years ago now, but, But yeah,
there was, yeah, I think MCAP

300
00:17:28,728 --> 00:17:32,503
kind of stuck out as, being,
philosophically similar to PCAP

301
00:17:32,503 --> 00:17:36,090
files, which are a network file
format that allows you to

302
00:17:36,090 --> 00:17:40,117
capture all of the packets going
out on your sort of IP address,

303
00:17:40,117 --> 00:17:44,207
through, a GCP IP stack. You can
capture all of these, all of the

304
00:17:44,207 --> 00:17:48,171
packets going forth and then you
can go and inspect that later.

305
00:17:48,171 --> 00:17:52,198
We're thinking this is not, This
is a higher abstraction, right?

306
00:17:52,198 --> 00:17:55,721
It's not at the level of
individual packets, it's at the

307
00:17:55,721 --> 00:17:59,560
level of messages, ROS messages
or other messages in a PubSub

308
00:17:59,560 --> 00:18:00,000
system.

309
00:18:00,000 --> 00:18:03,987
Audrow Nash: Nice. I like that.
And it also makes sense because

310
00:18:03,987 --> 00:18:08,038
maybe people familiar with PCAP
can map over their understanding

311
00:18:08,038 --> 00:18:10,000
and it's just, it's a different

312
00:18:10,000 --> 00:18:14,044
Adrian Macneil: yeah. It's,
yeah, it's, conceptually

313
00:18:14,044 --> 00:18:18,789
similar, although mostly it's
just a catchy name. And then at

314
00:18:18,789 --> 00:18:23,533
some point, some point, one of
the team members threw in the,

315
00:18:23,533 --> 00:18:28,200
the hat logo, caught on. And,
and that was a little bit of a

316
00:18:28,200 --> 00:18:31,000
dad joke that made it in with,
yeah.

317
00:18:31,000 --> 00:18:34,253
Audrow Nash: Okay. So we have
MCAP for recording. Where do we

318
00:18:34,253 --> 00:18:35,000
go from there?

319
00:18:35,000 --> 00:18:38,708
Adrian Macneil: got to get the
data off the robots. So this is

320
00:18:38,708 --> 00:18:42,417
how we think about as, upload
and ingestion. As I said, within

321
00:18:42,417 --> 00:18:46,245
the robotics space, usually you
are logging very, dense data and

322
00:18:46,245 --> 00:18:49,894
then you've almost universally
got a, pretty limited internet

323
00:18:49,894 --> 00:18:53,662
connection. bandwidth constraint
is just a universal problem in

324
00:18:53,662 --> 00:18:57,430
robotics. so how do we get the
data off the robot? during early

325
00:18:57,430 --> 00:19:01,199
R&amp; D, this is not, As, it's
not really as much of a problem

326
00:19:01,199 --> 00:19:04,907
people run into, they just, they
log in on the robot, probably

327
00:19:04,907 --> 00:19:08,556
the robot's right in front of
them, they, hook up an ethernet

328
00:19:08,556 --> 00:19:12,085
to the robot and SSH and copy
some files off, or, they just

329
00:19:12,085 --> 00:19:15,853
plug in a portable hard drive to
the robot and, off they go. As

330
00:19:15,853 --> 00:19:19,442
you start getting to robots in
the field, you start getting,

331
00:19:19,442 --> 00:19:22,971
the robots are not deployed
where your engineers are, where

332
00:19:22,971 --> 00:19:26,620
your team is, so you want to
start debugging data, you're not

333
00:19:26,620 --> 00:19:30,089
going to have like your entire
engineering team on site at

334
00:19:30,089 --> 00:19:33,738
wherever the actual robots are
being deployed, even the first

335
00:19:33,738 --> 00:19:37,566
few, and certainly once you get
to scale of thousands or tens of

336
00:19:37,566 --> 00:19:41,095
thousands of robots, you're
going to have them deployed all

337
00:19:41,095 --> 00:19:44,743
across the country, all across
the world, and you're going to

338
00:19:44,743 --> 00:19:48,272
have your engineering team is,
going to, possibly be in one

339
00:19:48,272 --> 00:19:51,263
place or possibly be
distributed, but probably not

340
00:19:51,263 --> 00:19:55,091
where your robots are. how do we
get the data off? first of all,

341
00:19:55,091 --> 00:19:58,560
it depends what, data you want
to get off. We see a lot of

342
00:19:58,560 --> 00:20:02,149
people, eventually move to some
kind of rolling record, type

343
00:20:02,149 --> 00:20:05,858
setup. So This, might involve
having a day or maybe max couple

344
00:20:05,858 --> 00:20:09,746
days worth of data logged on the
robot. You've got enough storage

345
00:20:09,746 --> 00:20:13,454
space on there for, 24 hours of
log data, say. But you're just

346
00:20:13,454 --> 00:20:16,923
overriding it, unless something
goes wrong, unless there's

347
00:20:16,923 --> 00:20:18,000
something notable.

348
00:20:18,000 --> 00:20:20,000
Audrow Nash: like a home
security system or

349
00:20:20,000 --> 00:20:23,719
Adrian Macneil: yeah, exactly,
yeah. It's if someone breaks

350
00:20:23,719 --> 00:20:27,312
into your house, you got to go
quickly pull the tapes out

351
00:20:27,312 --> 00:20:30,967
before, Yeah, or you're like a,
a black box recorder in an

352
00:20:30,967 --> 00:20:35,001
aircraft or something like that.
and then why, and then you, but

353
00:20:35,001 --> 00:20:38,972
you do want to capture the data
if it's, for example, something

354
00:20:38,972 --> 00:20:42,628
went wrong, obviously, like
robot, crashed into some, like

355
00:20:42,628 --> 00:20:46,662
packaged goods or the fell down
the stairs or whatever the robot

356
00:20:46,662 --> 00:20:50,192
was doing wrong. Those are
obvious ones. The other areas

357
00:20:50,192 --> 00:20:54,100
that you might want to capture
data are things going well. You

358
00:20:54,100 --> 00:20:57,630
want to just capture, a
distribution of data for your ML

359
00:20:57,630 --> 00:21:01,411
training, things like that. you
need, you want to get a nice

360
00:21:01,411 --> 00:21:04,752
distribution of, different
events. but we usually see

361
00:21:04,752 --> 00:21:08,660
people moving towards some kind
of, rolling record type setup.

362
00:21:08,660 --> 00:21:12,694
The exception to that, would be,
companies that are operating in

363
00:21:12,694 --> 00:21:16,287
public, so self driving cars,
but also things like cyborg

364
00:21:16,287 --> 00:21:20,321
delivery robots and things like
this, there is usually a higher,

365
00:21:20,321 --> 00:21:24,229
there's usually a desire with
those types of robots to save it

366
00:21:24,229 --> 00:21:28,011
for more than just what you
could fit locally, probably more

367
00:21:28,011 --> 00:21:31,856
than a day, right? You want to
have, Keep keeping most of the

368
00:21:31,856 --> 00:21:35,953
data around for a month or a few
months or something like this in

369
00:21:35,953 --> 00:21:39,924
case someone calls up and says
hey you cut me off on my bicycle

370
00:21:39,924 --> 00:21:43,705
I'm suing you or something like
this you want to not say you

371
00:21:43,705 --> 00:21:47,613
know it may be weeks later that
you find out about a potential

372
00:21:47,613 --> 00:21:49,000
incident so we usually

373
00:21:49,000 --> 00:21:52,875
Audrow Nash: they generate so
like autonomous cars at least so

374
00:21:52,875 --> 00:21:56,625
much data. So I imagine they
need to offload it because they

375
00:21:56,625 --> 00:21:58,000
can't store it because

376
00:21:58,000 --> 00:22:01,903
Adrian Macneil: yeah, so yeah,
yeah, so in the AV industry, and

377
00:22:01,903 --> 00:22:05,743
it's a maturity curve, you get
to the point that, that sort of

378
00:22:05,743 --> 00:22:09,646
like at Waymo is at today, and
you're absolutely going to see a

379
00:22:09,646 --> 00:22:13,487
rolling record and not saving
everything that's happening, but

380
00:22:13,487 --> 00:22:17,018
until you reach that level of
confidence, for, all of the

381
00:22:17,018 --> 00:22:20,239
early years, it's usually
something that looks like,

382
00:22:20,239 --> 00:22:24,018
logging to port of logging to
hotswap hard drives on the car.

383
00:22:24,018 --> 00:22:27,920
And then when the car is coming
back to charge or to fill up or

384
00:22:27,920 --> 00:22:31,699
be cleaned or whatever service,
we're just popping those hard

385
00:22:31,699 --> 00:22:35,540
drives out and, switching them
out for fresh ones. from there,

386
00:22:35,540 --> 00:22:39,381
of course, you still got this
just ridiculous quantity of data

387
00:22:39,381 --> 00:22:43,221
because, self driving car can be
logging multiple terabytes an

388
00:22:43,221 --> 00:22:47,062
hour easily. so you got to get
all of that data. Off a car and

389
00:22:47,062 --> 00:22:50,469
up to the cloud that requires a
pretty extreme internet

390
00:22:50,469 --> 00:22:54,186
connection. But yeah, that's the
AV use case is a little bit

391
00:22:54,186 --> 00:22:58,027
unique in that world. One where
you're operating in public and

392
00:22:58,027 --> 00:23:01,558
you're a lot more sensitive
things Potentially might have

393
00:23:01,558 --> 00:23:05,212
gone wrong that you might not
find out about. You don't see

394
00:23:05,212 --> 00:23:09,115
that as much in a warehouse AMR
or something like that. Usually

395
00:23:09,115 --> 00:23:13,142
the amount of damage that an AMR
can do driving at three miles an

396
00:23:13,142 --> 00:23:16,858
hour in a warehouse is fairly
limited. And also a lot of the

397
00:23:16,858 --> 00:23:20,327
time, a lot of these warehouse
AMRs have a lot of people

398
00:23:20,327 --> 00:23:23,673
around, they have buttons on
them where you can report

399
00:23:23,673 --> 00:23:27,575
incidents and things like this.
So you might be perfectly happy

400
00:23:27,575 --> 00:23:31,416
just saving a couple minutes.
maybe a minute or two minutes of

401
00:23:31,416 --> 00:23:34,575
data every time there's
something notable then just

402
00:23:34,575 --> 00:23:36,000
throwing the rest away.

403
00:23:36,000 --> 00:23:40,606
Audrow Nash: Okay. So back to
this data upload problem. what

404
00:23:40,606 --> 00:23:45,442
do you guys have in place? So I
guess there has been this, what

405
00:23:45,442 --> 00:23:50,048
people have done is they often
record the data in place and,

406
00:23:50,048 --> 00:23:54,807
or, but I guess if you want it
for machine learning and so you

407
00:23:54,807 --> 00:23:59,259
want to train on that data, you
want to get that data off.

408
00:23:59,259 --> 00:24:03,558
connectivity is very bad general
or not very bad, right?

409
00:24:03,558 --> 00:24:08,548
Generally, but often can be very
bad. so how are you guys solving

410
00:24:08,548 --> 00:24:13,000
this or what's your approach to
getting data off the robot

411
00:24:13,000 --> 00:24:16,496
Adrian Macneil: Yeah. exactly.
And so we offer, and I think

412
00:24:16,496 --> 00:24:20,228
there's, a lot more to build out
here over time, but we offer a

413
00:24:20,228 --> 00:24:23,961
agent that you can, we call the
robot agent. You can drop it on

414
00:24:23,961 --> 00:24:27,753
a robot. and then it'll watch a
directory of, log files, of MCAP

415
00:24:27,753 --> 00:24:31,545
files or bag files. So let's say
you're running a ROS robot or a

416
00:24:31,545 --> 00:24:35,278
ROS like robot. It's recording a
lot of data, to MCAP files, to

417
00:24:35,278 --> 00:24:38,951
ROS bags. those are typically
getting, the files typically get

418
00:24:38,951 --> 00:24:42,625
rotated every sort of 60 seconds
to a few minutes. Quite often

419
00:24:42,625 --> 00:24:46,180
people are splitting the files
up every few minutes just to,

420
00:24:46,180 --> 00:24:49,853
to, to make them easier to work
with. we have an agent that'll

421
00:24:49,853 --> 00:24:53,349
sit there and just watch that
directory. keep an eye on new

422
00:24:53,349 --> 00:24:56,904
files that are coming in. and it
notifies the server and the

423
00:24:56,904 --> 00:25:00,399
cloud about what files are
available. and then, we, have an

424
00:25:00,399 --> 00:25:04,014
understanding that, hey, you've
got this of robots out there,

425
00:25:04,014 --> 00:25:07,569
they have these, this data is
available for import, from the

426
00:25:07,569 --> 00:25:11,183
edge. And then, you can trigger
an upload in one of two ways.

427
00:25:11,183 --> 00:25:14,441
You can either do it from the
robot, so you can, send a

428
00:25:14,441 --> 00:25:17,937
message to the on, to the robot
agent, say, flag, flag this

429
00:25:17,937 --> 00:25:21,611
particular file, or say, can you
queue that for upload? or you

430
00:25:21,611 --> 00:25:24,988
can do it as a pull based
mechanism from the server, from

431
00:25:24,988 --> 00:25:28,602
the cloud. So I can go to the
web portal, I can click on that

432
00:25:28,602 --> 00:25:32,335
robot, I can see the timeline of
data that's available. I can't

433
00:25:32,335 --> 00:25:36,186
view it yet. I know that there's
data there, but I can't, I don't

434
00:25:36,186 --> 00:25:39,919
know what that data is. But I,
might, the, pull based workflow,

435
00:25:39,919 --> 00:25:43,533
the push based where the robot
initiates it is great when the

436
00:25:43,533 --> 00:25:47,147
robot identifies something wrong
because the robot identifies

437
00:25:47,147 --> 00:25:50,939
that there's been a collision or
the robot identifies some other

438
00:25:50,939 --> 00:25:54,257
kind of error that it wants to
report. It can trigger an

439
00:25:54,257 --> 00:25:57,990
upload. The pull based workflow,
where the user is going to the

440
00:25:57,990 --> 00:26:01,663
cloud and requesting it, is good
when they get a call from the

441
00:26:01,663 --> 00:26:05,218
operations team saying, Hey, at
2pm this afternoon there was

442
00:26:05,218 --> 00:26:08,832
this incident with the robot,
can you guys take a look at it?

443
00:26:08,832 --> 00:26:12,269
You might want to go in and
fetch, just grab 10 minutes of

444
00:26:12,269 --> 00:26:15,883
data around, around about when
a, some problem was claimed to

445
00:26:15,883 --> 00:26:19,438
happen, and then offload that,
pull it, From the, yeah, from

446
00:26:19,438 --> 00:26:23,111
the robot and have that sent to
the cloud so that you can then

447
00:26:23,111 --> 00:26:24,000
go investigate.

448
00:26:24,000 --> 00:26:28,296
Audrow Nash: Okay. So you're not
doing, it's not towards like

449
00:26:28,296 --> 00:26:29,000
dashboards

450
00:26:29,000 --> 00:26:32,695
Adrian Macneil: Yeah, no, we
we've yeah, exactly. We've,

451
00:26:32,695 --> 00:26:36,654
stayed away from that. there are
a lot of tools that do that

452
00:26:36,654 --> 00:26:40,679
already. you've got, Foment is
probably one of the most known

453
00:26:40,679 --> 00:26:44,638
ones, but, in all, but there's,
there's a lot of these tools

454
00:26:44,638 --> 00:26:48,598
that are creating, more of a
live overview. I call it like a

455
00:26:48,598 --> 00:26:52,887
management platform. we don't do
anything in the fleet management

456
00:26:52,887 --> 00:26:56,978
space. it's, a little bit It's a
different problem category, I

457
00:26:56,978 --> 00:27:00,937
think, to what we're solving,
we're focusing on the needs of

458
00:27:00,937 --> 00:27:04,962
robotics engineers, robotics
developers, and getting, what is

459
00:27:04,962 --> 00:27:08,921
usually, async access to raw,
detailed logs about incidents,

460
00:27:08,921 --> 00:27:12,550
versus a fleet overview, which
is more of an operations

461
00:27:12,550 --> 00:27:16,707
workflow of just hey, I want to
see where my robots are. I want

462
00:27:16,707 --> 00:27:20,798
to click on this one and send
this one home. I want to see how

463
00:27:20,798 --> 00:27:24,823
many, tasks per hour are being
completed by different robots.

464
00:27:24,823 --> 00:27:28,387
this kind of stuff is, yeah,
it's, a different kind of

465
00:27:28,387 --> 00:27:32,148
product category and problem
space. It's adjacent to what

466
00:27:32,148 --> 00:27:35,843
we're doing, but it's not
something that we have focused

467
00:27:35,843 --> 00:27:40,000
on now and we're unlikely to be
focusing on in the near future.

468
00:27:40,000 --> 00:27:43,578
Audrow Nash: Okay. So that
sounds cool. So you basically,

469
00:27:43,578 --> 00:27:47,596
you have something that you have
your agent is watching a folder

470
00:27:47,596 --> 00:27:51,489
and it's letting you know when
there's something there and you

471
00:27:51,489 --> 00:27:55,193
can pull it from wherever,
whatever computer you're on when

472
00:27:55,193 --> 00:27:59,148
you're not close to the robot or
will push it or it can push it

473
00:27:59,148 --> 00:28:03,040
periodically. and it's really
just, you're going to upload the

474
00:28:03,040 --> 00:28:06,933
whole thing when it's pushed or
pulled, to wherever it is that

475
00:28:06,933 --> 00:28:08,000
it should be sent

476
00:28:08,000 --> 00:28:11,067
Adrian Macneil: Yeah. And
snippets of data. there's

477
00:28:11,067 --> 00:28:14,135
different ways to configure
this. Sometimes people,

478
00:28:14,135 --> 00:28:17,503
sometimes another, pattern see
sometimes is, people will

479
00:28:17,503 --> 00:28:21,232
actually log two separate sets
of, MCAP files, on their robot.

480
00:28:21,232 --> 00:28:24,961
They'll be logging one that's
all the lightweight telemetry on

481
00:28:24,961 --> 00:28:28,449
their robot. So things like just
the robot pose, maybe GPS

482
00:28:28,449 --> 00:28:31,516
position, system state, just
stuff that's like very

483
00:28:31,516 --> 00:28:35,306
lightweight, and then a separate
file that's logging all of the

484
00:28:35,306 --> 00:28:38,493
heavyweight sensor data, the
video, the point clouds,

485
00:28:38,493 --> 00:28:42,102
anything like this. the beauty
of that setup is that you can

486
00:28:42,102 --> 00:28:45,951
actually just always upload the
lightweight stuff. So you always

487
00:28:45,951 --> 00:28:49,620
get some amount of detail, not
immediately because it's still

488
00:28:49,620 --> 00:28:53,349
an async batch upload workflow,
but at some point within maybe

489
00:28:53,349 --> 00:28:56,837
10 minutes of, real time, five,
five to 10 minutes, you're

490
00:28:56,837 --> 00:29:00,686
seeing enough data that you can,
figure out where a robot was at

491
00:29:00,686 --> 00:29:04,175
any point in time. That might
help you narrow down on, say

492
00:29:04,175 --> 00:29:08,024
again, someone calls you up and
says there was a problem at 2 p.

493
00:29:08,024 --> 00:29:11,272
m. That will help you narrow
down on exactly when that

494
00:29:11,272 --> 00:29:14,700
problem was and roughly where
the robot was. Then you can

495
00:29:14,700 --> 00:29:18,369
select a time range and go and
download and upload the, fetch

496
00:29:18,369 --> 00:29:21,797
the more sort of detailed HD
logs. So that's quite a nice

497
00:29:21,797 --> 00:29:23,000
pattern that we see,

498
00:29:23,000 --> 00:29:26,000
Audrow Nash: that is a really
cool one.

499
00:29:26,000 --> 00:29:29,023
Adrian Macneil: But yeah, you do
all of that, and then your data

500
00:29:29,023 --> 00:29:31,998
gets to the cloud, right? Yeah.
and again, all of these, one of

501
00:29:31,998 --> 00:29:35,021
the philosophies of how we build
Foxglove is trying to make this

502
00:29:35,021 --> 00:29:37,618
really modular, because not
everyone wants all of these

503
00:29:37,618 --> 00:29:40,310
pieces, right? Some people have
already figured out their

504
00:29:40,310 --> 00:29:43,049
logging, some people have
already figured out their upload

505
00:29:43,049 --> 00:29:46,025
workflows, they don't need help
with that. but, others haven't.

506
00:29:46,025 --> 00:29:49,000
We just say, pick and choose the
components that make sense for

507
00:29:49,000 --> 00:29:52,776
Audrow Nash: Yeah. So you
support the full vertical of

508
00:29:52,776 --> 00:29:57,112
like I record my data to, I do
an analysis on my data. but, or

509
00:29:57,112 --> 00:30:01,517
whatever it might be, I use it
for training. but you can use it

510
00:30:01,517 --> 00:30:05,853
for only specific parts. So if
just want to upload the data or

511
00:30:05,853 --> 00:30:09,000
if you just want to visualize
the data or you

512
00:30:09,000 --> 00:30:12,048
Adrian Macneil: yeah, if you
just care about uploading and

513
00:30:12,048 --> 00:30:15,097
organizing your data in the
cloud, that's a, thing you can

514
00:30:15,097 --> 00:30:18,197
do. Typically we find the, you
get to the sort of last part

515
00:30:18,197 --> 00:30:20,878
we'll get to around
visualization analysis. Usually

516
00:30:20,878 --> 00:30:24,031
that, those workflows are pretty
common across everyone. but

517
00:30:24,031 --> 00:30:27,290
yeah, absolutely a lot of People
have come to us and initially

518
00:30:27,290 --> 00:30:30,443
wanted to focus on the data
workflows and then they get into

519
00:30:30,443 --> 00:30:33,229
the visualization. Sometimes
it's like start with the

520
00:30:33,229 --> 00:30:36,435
visualization. I'm happy with
how I get data off the robot. I

521
00:30:36,435 --> 00:30:39,536
just want to visualize it.
Great. That's fine. We can start

522
00:30:39,536 --> 00:30:42,584
at that end. but yeah, as we're
getting all of this, we're

523
00:30:42,584 --> 00:30:45,685
triggering these uploads, we're
getting logs coming off the

524
00:30:45,685 --> 00:30:46,000
robot.

525
00:30:46,000 --> 00:30:49,727
Adrian Macneil: then, you get to
the data in the cloud and again,

526
00:30:49,727 --> 00:30:52,537
We see a lot of, robotics
companies, at least pre

527
00:30:52,537 --> 00:30:56,035
Foxglove, was just, there's an
S3 bucket, we've got an Amazon

528
00:30:56,035 --> 00:30:59,705
S3 bucket, we've dumped a bunch
of bag files in it, or something

529
00:30:59,705 --> 00:31:03,088
like that. so if you know what
file, you know the file name

530
00:31:03,088 --> 00:31:06,643
you're looking for, great, but
if you don't, good luck to you.

531
00:31:06,643 --> 00:31:10,313
so that was, the, the problem we
set out to solve with the cloud

532
00:31:10,313 --> 00:31:13,524
indexing was, Okay, as these
files are coming in, let's,

533
00:31:13,524 --> 00:31:17,252
let's do an ingestion step where
we look at metadata on the file.

534
00:31:17,252 --> 00:31:20,692
We look at what time range is
being covered by this file. We

535
00:31:20,692 --> 00:31:24,362
look at what robot it came off.
what topics are inside the file.

536
00:31:24,362 --> 00:31:27,745
we go through this ingestion
step, and we store all of that

537
00:31:27,745 --> 00:31:31,186
in a nicely organized indexed,
bucket. Either we can, either

538
00:31:31,186 --> 00:31:34,684
Fox Club hosted or it can be
hosted in your own, your own VPC

539
00:31:34,684 --> 00:31:38,354
with any cloud provider. but we,
go through this ingestion step.

540
00:31:38,354 --> 00:31:41,966
We organize it into a nice sort
of folder structure within your

541
00:31:41,966 --> 00:31:45,292
bucket. And then we also give
you a web UI so that you can

542
00:31:45,292 --> 00:31:48,733
search across it, right? You
can, just go in and type in and

543
00:31:48,733 --> 00:31:52,288
search metadata. You can look at
a timeline view. You can do a

544
00:31:52,288 --> 00:31:55,499
few more of these things to, a
more, sort of like visual

545
00:31:55,499 --> 00:31:58,997
overview. if you want to look
up, you know which robot you're

546
00:31:58,997 --> 00:32:02,380
looking for, and you know the
time range, or you don't know

547
00:32:02,380 --> 00:32:05,706
the name of the file, it's much
easier to find the sort of

548
00:32:05,706 --> 00:32:08,000
particular logs that you're
looking for.

549
00:32:08,000 --> 00:32:11,625
Audrow Nash: Yeah, I feel like
this is something that's so

550
00:32:11,625 --> 00:32:13,000
simple and so valuable

551
00:32:13,000 --> 00:32:16,335
Adrian Macneil: yeah, and
everyone has, and, I don't claim

552
00:32:16,335 --> 00:32:19,786
a lot of the stuff that we do,
it's not rocket science, it's

553
00:32:19,786 --> 00:32:23,179
just really undifferentiated,
because you see every company

554
00:32:23,179 --> 00:32:26,744
build a web UI for looking at
ROS bags or something like this,

555
00:32:26,744 --> 00:32:30,195
and you're like, this is just
silly, everyone shouldn't need

556
00:32:30,195 --> 00:32:31,000
to build this.

557
00:32:31,000 --> 00:32:33,000
Audrow Nash: Yeah, for

558
00:32:33,000 --> 00:32:34,000
Adrian Macneil: yeah,

559
00:32:34,000 --> 00:32:38,168
Audrow Nash: And then a UI for
filtering through the data.

560
00:32:38,168 --> 00:32:42,120
yeah, I'm seeing a lot of,
analog to like home security

561
00:32:42,120 --> 00:32:46,000
with this, 'cause you can look
for this camera at this

562
00:32:46,000 --> 00:32:47,000
Adrian Macneil: right, pull

563
00:32:47,000 --> 00:32:51,041
Audrow Nash: look for this robot
from this time window. and you

564
00:32:51,041 --> 00:32:54,953
might be able to filter by a
specific event or something that

565
00:32:54,953 --> 00:32:58,994
occurred, but so you're pulling
out metadata. By is very simple

566
00:32:58,994 --> 00:33:02,907
by processing, what has been
uploaded. You get the start time

567
00:33:02,907 --> 00:33:06,948
and time. Maybe you key or tag
it with things like this is this

568
00:33:06,948 --> 00:33:09,000
robot in this warehouse this and

569
00:33:09,000 --> 00:33:12,697
Adrian Macneil: Yeah, can, yeah,
tag all of that assign it to

570
00:33:12,697 --> 00:33:16,576
what we call a device, which is
your robot, and then you're able

571
00:33:16,576 --> 00:33:20,152
to view all of that data on a
timeline, or you'd be able to

572
00:33:20,152 --> 00:33:23,061
pull up just the logs for
particular. just for a

573
00:33:23,061 --> 00:33:26,818
particular robot, or if if you
are tagging them with metadata,

574
00:33:26,818 --> 00:33:30,697
such as which factory or, which
site it was operating at, tag it

575
00:33:30,697 --> 00:33:34,333
by that kind of metadata as
well. but you've got this, this,

576
00:33:34,333 --> 00:33:37,000
kind of quick search across all
of the data.

577
00:33:37,000 --> 00:33:40,533
Audrow Nash: it. Okay. I think
that's really nice. and that

578
00:33:40,533 --> 00:33:43,886
feels like something where
companies might plot along in

579
00:33:43,886 --> 00:33:47,000
their like terrible naming
convention. Like the file

580
00:33:47,000 --> 00:33:49,000
Adrian Macneil: It's got the
robot

581
00:33:49,000 --> 00:33:53,036
Audrow Nash: where It's this is
the robot. Yeah. And then, that

582
00:33:53,036 --> 00:33:57,135
is probably, it works, but it's
super painful. especially as you

583
00:33:57,135 --> 00:34:01,107
get a lot of robots and, a lot
of data. And then this makes it

584
00:34:01,107 --> 00:34:04,758
so much easier. And then you
have the UI, which is really

585
00:34:04,758 --> 00:34:07,000
nice for filtering and things
okay.

586
00:34:07,000 --> 00:34:10,393
Adrian Macneil: a lot more that
we can do there too, today we

587
00:34:10,393 --> 00:34:13,896
let you search by metadata that
you've pre added, some of these

588
00:34:13,896 --> 00:34:17,456
things like you said by robot or
by, other metadata you had like

589
00:34:17,456 --> 00:34:20,793
factory and things, but there's
a lot of other ways that you

590
00:34:20,793 --> 00:34:24,074
could imagine exploring this
data, one we would like to add

591
00:34:24,074 --> 00:34:27,633
in the future would be a sort of
a geospatial search over, can I

592
00:34:27,633 --> 00:34:31,137
see all of my data on a map if
you just see like a heat map and

593
00:34:31,137 --> 00:34:34,696
then say in and okay, I want to
pull all of the logs, all of the

594
00:34:34,696 --> 00:34:38,033
times that we've been past,
whether it's an indoor map or an

595
00:34:38,033 --> 00:34:41,537
outdoor map, it's not, similar
kind concept, but I want to pull

596
00:34:41,537 --> 00:34:44,763
all of the logs that I've, or
all of the times that robots

597
00:34:44,763 --> 00:34:48,266
have been to this corner of the
factory or they've been through

598
00:34:48,266 --> 00:34:51,770
this intersection or things like
that, those types of searches,

599
00:34:51,770 --> 00:34:55,051
are things that we would like to
add the future to make it,

600
00:34:55,051 --> 00:34:58,333
really easy to, pin down on the
particular data that you're

601
00:34:58,333 --> 00:34:59,000
looking for.

602
00:34:59,000 --> 00:35:02,658
Audrow Nash: Yeah, I feel like
you have, there's a lot, I mean

603
00:35:02,658 --> 00:35:06,434
you could do a lot of analysis,
you could do a lot of displaying

604
00:35:06,434 --> 00:35:10,032
stuff, it's, the ceiling's the
limit for what you could do to

605
00:35:10,032 --> 00:35:13,277
expose different data things
there, but that seems very

606
00:35:13,277 --> 00:35:16,876
valuable. Spatial seems super
valuable, specific events could

607
00:35:16,876 --> 00:35:19,000
be valuable, having them write
their

608
00:35:19,000 --> 00:35:22,217
Adrian Macneil: Yeah. Yeah.
Yeah. We do it. Yeah. do have an

609
00:35:22,217 --> 00:35:25,166
event. Yeah. We do an event
concept. So you can whether

610
00:35:25,166 --> 00:35:28,598
whether those are being created
through the API from things that

611
00:35:28,598 --> 00:35:31,869
are happening on the robot or
whether it's being created, you

612
00:35:31,869 --> 00:35:34,710
can just go into the UI and
annotate your own events,

613
00:35:34,710 --> 00:35:37,874
manually, but tagging, tagging
points in time where notable

614
00:35:37,874 --> 00:35:39,000
things happening. And

615
00:35:39,000 --> 00:35:43,369
Audrow Nash: the super cool.
Okay. So we have this cloud

616
00:35:43,369 --> 00:35:47,972
indexing, so you're able to get
your data very effectively.

617
00:35:47,972 --> 00:35:50,000
where do we go from there?

618
00:35:50,000 --> 00:35:53,000
Adrian Macneil: Analysis and, in
particular visualization.

619
00:35:53,000 --> 00:35:55,813
Adrian Macneil: so visualization
is, like I said, this is where

620
00:35:55,813 --> 00:35:58,000
we started the end of the
problem that we started

621
00:35:58,000 --> 00:36:01,297
Audrow Nash: Yeah, I think in
our last interview, which might

622
00:36:01,297 --> 00:36:02,000
have been two

623
00:36:02,000 --> 00:36:03,000
Adrian Macneil: was a it while
ago. Yeah.

624
00:36:03,000 --> 00:36:07,357
Audrow Nash: point it was most
of what you had was the visual

625
00:36:07,357 --> 00:36:11,714
visualization and you probably
were working on the cloud part

626
00:36:11,714 --> 00:36:15,714
and it was pre mcap if i
remember correctly so okay yeah

627
00:36:15,714 --> 00:36:16,000
time

628
00:36:16,000 --> 00:36:19,783
Adrian Macneil: Yeah, it does.
Yeah. a lot has been built in

629
00:36:19,783 --> 00:36:23,502
the past, past few years. but
yeah, so we started with this

630
00:36:23,502 --> 00:36:27,096
visualization. and again,
another difference between sort

631
00:36:27,096 --> 00:36:30,942
of observability with, servers
and traditional infrastructure

632
00:36:30,942 --> 00:36:35,040
versus robotics, and traditional
infrastructure, again, you think

633
00:36:35,040 --> 00:36:38,760
of a tool like a Datadog or
Grafana or something like this,

634
00:36:38,760 --> 00:36:42,605
you're creating a lot of nice
pretty plots and dashboards and

635
00:36:42,605 --> 00:36:46,136
things, but usually only with
data that's already in the

636
00:36:46,136 --> 00:36:50,108
cloud. this is, why the, the arc
of data goes from like logging

637
00:36:50,108 --> 00:36:53,828
to get it to the cloud, to
indexing, to visualizing it. but

638
00:36:53,828 --> 00:36:57,800
in robotics, visualizing cloud
data is only one of, I would say

639
00:36:57,800 --> 00:37:01,582
three kind of common workflows,
right? So one is visualizing

640
00:37:01,582 --> 00:37:04,861
cloud data. Another is
visualizing local files. I've

641
00:37:04,861 --> 00:37:08,896
just got, some files that I've,
off or, SCP'd off a robot or not

642
00:37:08,896 --> 00:37:12,805
uploaded or I'm around locally.
another is live visualization,

643
00:37:12,805 --> 00:37:16,840
so I'm this could be either, I'm
connected to a robot over Wi Fi

644
00:37:16,840 --> 00:37:20,875
or Ethernet, and, I'm literally
just plugged into a, robot and I

645
00:37:20,875 --> 00:37:24,720
want to see the data coming in
live and I want to check that,

646
00:37:24,720 --> 00:37:28,629
the colorations are correct and
that the data makes sense, or,

647
00:37:28,629 --> 00:37:32,349
another, option with the live
one is if you're running some

648
00:37:32,349 --> 00:37:36,258
kind of re simulation or some
kind of simulation on your, just

649
00:37:36,258 --> 00:37:40,230
on your local workstation, your
local laptop or desktop, and so

650
00:37:40,230 --> 00:37:43,697
you've got a robot stack
running, you might not have an

651
00:37:43,697 --> 00:37:47,732
actual sort of full, hardware in
the loop or anything, it's just

652
00:37:47,732 --> 00:37:51,641
running, a stack locally, but
you still want to be able to see

653
00:37:51,641 --> 00:37:55,550
the, output of, what your stack
thinks is happening, what your

654
00:37:55,550 --> 00:37:57,000
stack thinks is seeing.

655
00:37:57,000 --> 00:38:01,000
Audrow Nash: i'm thinking very
similar to RViz this for

656
00:38:01,000 --> 00:38:04,338
Adrian Macneil: yeah. And Rviz,
really only supports the, live

657
00:38:04,338 --> 00:38:07,568
visualization workflow. So when
you want to visualize things

658
00:38:07,568 --> 00:38:10,852
with Rviz, you can connect to a
live running ROS. You can see

659
00:38:10,852 --> 00:38:14,028
that in real time. But if you
want to go and replay, if you

660
00:38:14,028 --> 00:38:17,204
want to go and explore a BAG
file or MCAP file, You have to

661
00:38:17,204 --> 00:38:20,434
separate, you have to go start
up ROS in the background. You

662
00:38:20,434 --> 00:38:23,287
have to start up, run ROSBag
play. So you've got five

663
00:38:23,287 --> 00:38:26,356
different kind of terminal
windows in the background with

664
00:38:26,356 --> 00:38:29,693
your ROS stack running and your
play running. And, then if you

665
00:38:29,693 --> 00:38:32,762
want to pause and rewind
something, you've got to do that

666
00:38:32,762 --> 00:38:34,000
sort of in the terminal

667
00:38:34,000 --> 00:38:36,000
Audrow Nash: it's a pain.

668
00:38:36,000 --> 00:38:39,197
Adrian Macneil: which is so
there's no, with something like

669
00:38:39,197 --> 00:38:42,665
Rviz, you have no native support
for a kind of, for a file based

670
00:38:42,665 --> 00:38:45,537
or a cloud based, replay
workflow. You only have this

671
00:38:45,537 --> 00:38:47,000
kind of live visualization.

672
00:38:47,000 --> 00:38:51,795
Audrow Nash: Now, one, one thing
with RViz is it's running on

673
00:38:51,795 --> 00:38:56,039
your local machine. I know
Foxglove can be but running

674
00:38:56,039 --> 00:39:00,755
through Electron or however
you're displaying it these days.

675
00:39:00,755 --> 00:39:05,000
but so RViz might be more
performant, would you think?

676
00:39:05,000 --> 00:39:07,276
Audrow Nash: So if data is
coming in really fast and you

677
00:39:07,276 --> 00:39:09,593
want to display, or does it not
matter because it's human

678
00:39:09,593 --> 00:39:10,000
perception

679
00:39:10,000 --> 00:39:13,000
Adrian Macneil: Yeah, it really
depends, yeah,

680
00:39:13,000 --> 00:39:16,000
Audrow Nash: how do you think
about this?

681
00:39:16,000 --> 00:39:19,152
Adrian Macneil: it, depends a
lot, right? in terms of, the

682
00:39:19,152 --> 00:39:22,521
rendering, our 3D rendering and
things like that's all running

683
00:39:22,521 --> 00:39:25,727
through WebGL. a lot of the
things, a lot of the things are

684
00:39:25,727 --> 00:39:28,715
actually already hard or
accelerated. A lot of the code

685
00:39:28,715 --> 00:39:31,650
in a browser environment, people
think about a browser

686
00:39:31,650 --> 00:39:34,747
environment, The browser has
evolved a lot in the past 10

687
00:39:34,747 --> 00:39:37,790
years, right? You've got
companies like Figma and things

688
00:39:37,790 --> 00:39:40,942
that are really pushing the
boundaries of what's possible.

689
00:39:40,942 --> 00:39:44,311
you've got services like we're
using now, services like Google

690
00:39:44,311 --> 00:39:47,463
Meet, things like this where
they're doing like live video

691
00:39:47,463 --> 00:39:50,614
and streaming and things in the
browser. Most of that code

692
00:39:50,614 --> 00:39:53,983
that's running is C code, right?
Like most of it is running in

693
00:39:53,983 --> 00:39:57,189
the background of the browser
stack that's implemented in C

694
00:39:57,189 --> 00:40:00,341
and you're just triggering it
from JavaScript. the same is

695
00:40:00,341 --> 00:40:03,384
true with a lot of this 3D
rendering, right? So a lot of

696
00:40:03,384 --> 00:40:06,699
the stuff that's happening in
Foxglove when you're looking at

697
00:40:06,699 --> 00:40:09,796
a 3D, View, like you would see
in Rviz, you are, that is.

698
00:40:09,796 --> 00:40:13,002
that's running through WebGL,
which is then being, like, in

699
00:40:13,002 --> 00:40:15,665
most cases, hardware
accelerated. Same when we do

700
00:40:15,665 --> 00:40:18,980
things like H. 264, like video
decoding and things like that.

701
00:40:18,980 --> 00:40:22,457
There are actually, these days,
there are browser APIs that will

702
00:40:22,457 --> 00:40:25,826
let you use hardware accelerated
decoding for video and things

703
00:40:25,826 --> 00:40:28,000
like that. So we leverage a lot
of that.

704
00:40:28,000 --> 00:40:31,333
Audrow Nash: That's awesome.
Yeah, it's so cool. It's like a

705
00:40:31,333 --> 00:40:34,889
rising tide lifts all boats. You
get to benefit from the massive

706
00:40:34,889 --> 00:40:37,000
investments going on into the
browser.

707
00:40:37,000 --> 00:40:40,327
Adrian Macneil: Yeah, and it's a
great platform, right? the

708
00:40:40,327 --> 00:40:43,992
browser is very extensible, it's
very easy to develop with, it's,

709
00:40:43,992 --> 00:40:46,924
we can let people, it's
pluggable, right? We can let

710
00:40:46,924 --> 00:40:50,081
people create, plugins and
things very easily, or create

711
00:40:50,081 --> 00:40:53,408
little forms and things like
that, that they want. and then

712
00:40:53,408 --> 00:40:56,904
also, the other thing with Rviz
is, this sort of, mostly gives

713
00:40:56,904 --> 00:41:00,343
you the 3D panel, right? If you
want to do things like plots,

714
00:41:00,343 --> 00:41:04,008
you're popping out to a separate
like RQT or your plot juggler or

715
00:41:04,008 --> 00:41:07,617
things like that. you don't you
can't just bring it, or maps and

716
00:41:07,617 --> 00:41:11,000
things like this, you can't
bring it all into one, into one.

717
00:41:11,000 --> 00:41:14,000
Audrow Nash: a bunch of windows
open for

718
00:41:14,000 --> 00:41:17,457
Adrian Macneil: but the
performance, the performance

719
00:41:17,457 --> 00:41:21,645
thing is, not a, a major thing
that comes up. It really depends

720
00:41:21,645 --> 00:41:25,501
on what people's data and
workflows look like. And there's

721
00:41:25,501 --> 00:41:29,291
such a variety within, within
the robotic space that you,

722
00:41:29,291 --> 00:41:32,881
just, it's, hard to say
concretely like X works better

723
00:41:32,881 --> 00:41:37,003
than Y, without looking at type
of data that people are trying

724
00:41:37,003 --> 00:41:38,000
to use it with.

725
00:41:38,000 --> 00:41:41,452
Audrow Nash: very Was this the
case before, say three, two

726
00:41:41,452 --> 00:41:45,083
years ago or something when we
talked? Because I remember, or

727
00:41:45,083 --> 00:41:48,536
maybe it's just the general
perception that because it's a

728
00:41:48,536 --> 00:41:52,107
web app, it can't have good
performance. and it's surprising

729
00:41:52,107 --> 00:41:55,560
to me and very nice to hear,
that you're getting wonderful

730
00:41:55,560 --> 00:41:58,000
performance and it's hardware
accelerated

731
00:41:58,000 --> 00:42:02,372
Adrian Macneil: yeah. It is, I
will say there's like a lot of

732
00:42:02,372 --> 00:42:06,673
stigma around, out there around
kind of Electron apps things

733
00:42:06,673 --> 00:42:11,332
like that. but I think in, some,
in some cases Electron is like a

734
00:42:11,332 --> 00:42:15,203
victim of its own success,
right? it's, it's created a

735
00:42:15,203 --> 00:42:18,930
platform that, is, so easy to
create. Cross platform

736
00:42:18,930 --> 00:42:23,159
applications with, without
having to go into, some of these

737
00:42:23,159 --> 00:42:27,030
more esoteric, yeah, like
problems that, yeah, have on

738
00:42:27,030 --> 00:42:31,402
specific problems, for specific
platforms. There are actually

739
00:42:31,402 --> 00:42:35,129
some, sort of, Other
alternatives that have come out

740
00:42:35,129 --> 00:42:39,000
more recently, there's one
called Tauri, or Tauri, I'm

741
00:42:39,000 --> 00:42:40,000
Audrow Nash: looks

742
00:42:39,000 --> 00:42:42,984
Adrian Macneil: Tauri. So this
this is a middle ground, right?

743
00:42:42,984 --> 00:42:46,711
It's it's still letting you use
the web stack to create an

744
00:42:46,711 --> 00:42:50,696
application, but it's using the,
OS specific built in web view

745
00:42:50,696 --> 00:42:54,873
components. the nice thing about
that is, is, it reduces the file

746
00:42:54,873 --> 00:42:58,664
size that you're downloading and
having to ship a, specific

747
00:42:58,664 --> 00:43:02,777
version of Chromium, that build.
this, that is also its biggest.

748
00:43:02,777 --> 00:43:06,568
Downfall, right? Because the
nice thing about, the way that

749
00:43:06,568 --> 00:43:10,167
we package and distribute
Foxglove is that every, we can

750
00:43:10,167 --> 00:43:13,830
guarantee that you're, it
doesn't matter you're on Ubuntu

751
00:43:13,830 --> 00:43:17,428
18 or 20 Windows or Mac or
Windows 10, Windows 11. We're

752
00:43:17,428 --> 00:43:21,348
not trying to guess at what a
web viewer, a native web viewer

753
00:43:21,348 --> 00:43:25,204
is doing. we're not dealing with
the problems like Safari is

754
00:43:25,204 --> 00:43:29,253
actually like really behind on a
lot of, edge web features, for

755
00:43:29,253 --> 00:43:33,044
example. We're not dealing with
hey, this problem that only

756
00:43:33,044 --> 00:43:36,000
seems to happen on Mac. the,
very occasionally

757
00:43:36,000 --> 00:43:40,991
Audrow Nash: better for you guys
as developers too. You have far

758
00:43:40,991 --> 00:43:45,046
less maintenance burden. and
like ROS, we, just as a

759
00:43:45,046 --> 00:43:50,037
developer on the robot operating
system, it's every time there's

760
00:43:50,037 --> 00:43:53,000
a problem on Windows, no one
wants it.

761
00:43:53,000 --> 00:43:56,090
Adrian Macneil: No yeah, And
then also it's good luck if

762
00:43:56,090 --> 00:43:59,510
you're on a Mac, it's basically
unsupported. they call it tier

763
00:43:59,510 --> 00:44:01,000
three something, but, yeah,

764
00:44:01,000 --> 00:44:03,000
Audrow Nash: build from source,
but we don't

765
00:44:03,000 --> 00:44:06,743
Adrian Macneil: exactly. Yeah,
yeah. So this is, these are huge

766
00:44:06,743 --> 00:44:10,366
benefits, especially when you're
a small team trying to focus

767
00:44:10,366 --> 00:44:13,990
building a great product. I will
say performance, and, issues

768
00:44:13,990 --> 00:44:17,554
come up from time to time and
customers bring them to us and

769
00:44:17,554 --> 00:44:21,000
it's usually a case of looking
at just specifically what's

770
00:44:21,000 --> 00:44:24,683
different about their data and
how can we, improve that. It is

771
00:44:24,683 --> 00:44:28,188
a thing that, that we spend
ongoing effort on, and if folks

772
00:44:28,188 --> 00:44:31,574
do have problems, we encourage
them to bring them forward

773
00:44:31,574 --> 00:44:34,842
because it's, like I said,
sometimes it's, it's hard to

774
00:44:34,842 --> 00:44:38,228
know if someone has a, an issue
like where exactly that's

775
00:44:38,228 --> 00:44:39,000
arising from.

776
00:44:39,000 --> 00:44:42,894
Audrow Nash: Definitely. Okay.
So we have going back to

777
00:44:42,894 --> 00:44:47,000
visualizations. Okay. And it is
an Electron app, which you

778
00:44:47,000 --> 00:44:50,617
Adrian Macneil: correct. Yeah,
we, built it as a, yeah, built

779
00:44:50,617 --> 00:44:54,055
it as a, web application. most
of our users are on desktop

780
00:44:54,055 --> 00:44:57,731
actually, I think from memory,
it's, like a majority of people

781
00:44:57,731 --> 00:45:01,466
are using the desktop app more
so than the, than actually using

782
00:45:01,466 --> 00:45:02,000
it in the

783
00:45:02,000 --> 00:45:04,000
Audrow Nash: through the
browser,

784
00:45:04,000 --> 00:45:07,167
Adrian Macneil: about the
desktop app can just install it

785
00:45:07,167 --> 00:45:09,000
and it works on any platform.
but

786
00:45:09,000 --> 00:45:11,000
Audrow Nash: cool.

787
00:45:11,000 --> 00:45:14,740
Adrian Macneil: so, yeah, we,
have, that, and I guess, we're

788
00:45:14,740 --> 00:45:18,729
saying this word visualization,
which I assume to like most sort

789
00:45:18,729 --> 00:45:21,846
of roboticists is a sort of
familiar concept, but,

790
00:45:21,846 --> 00:45:25,461
surprisingly, surprisingly
unfamiliar outside of robotics,

791
00:45:25,461 --> 00:45:28,764
right? People you go to an
average VC and say we make

792
00:45:28,764 --> 00:45:32,255
robotics visualization tools.
you'll get, a blank stare,

793
00:45:32,255 --> 00:45:34,000
right? this is not a, a that

794
00:45:34,000 --> 00:45:36,000
Audrow Nash: what do you need to
visualize?

795
00:45:36,000 --> 00:45:39,385
Adrian Macneil: Right, what do
you mean? Who would use that?

796
00:45:39,385 --> 00:45:42,487
Why they, get all these
questions like, why would I use

797
00:45:42,487 --> 00:45:45,815
that a robotics developer? I
don't even know how to explain

798
00:45:45,815 --> 00:45:47,000
it. I was like, fact,

799
00:45:47,000 --> 00:45:51,220
Audrow Nash: to realize how much
in our bubble we are the idea

800
00:45:51,220 --> 00:45:55,304
that's not clear is, surprising
to me too. And I suppose you

801
00:45:55,304 --> 00:45:59,047
talking to VCs, this is
something that's really come up

802
00:45:59,047 --> 00:46:00,000
they're out of

803
00:46:00,000 --> 00:46:03,367
Adrian Macneil: Yeah. Yeah, that
and, yeah, because, every

804
00:46:03,367 --> 00:46:06,965
industry has its, sort of, its,
uniquenesses, but I think, as,

805
00:46:06,965 --> 00:46:10,274
especially as you look at,
investors and things, a lot of

806
00:46:10,274 --> 00:46:13,815
the times, industries are a bit
more mature and they're a bit

807
00:46:13,815 --> 00:46:17,181
more mapped out and they can go
and see these little, logo

808
00:46:17,181 --> 00:46:20,780
clouds and understand how all
the things work together. Pieces

809
00:46:20,780 --> 00:46:24,263
fit in and they Oh, you do a
thing in XYZ category. And then

810
00:46:24,263 --> 00:46:27,746
you pop up in, in a category
that, that's not represented on

811
00:46:27,746 --> 00:46:31,229
their little logo cloud. And,
you have to explain from, sort

812
00:46:31,229 --> 00:46:34,711
of first principles. But I do, I
I, do think, and if anyone,

813
00:46:34,711 --> 00:46:37,904
listening is not a robotics
developer, visualization is

814
00:46:37,904 --> 00:46:41,328
something that is critical to
robotics development from day

815
00:46:41,328 --> 00:46:45,043
one of your very first robotics
hobby project. We're working on,

816
00:46:45,043 --> 00:46:48,062
autonomous robots in space or
autonomous vehicles or

817
00:46:48,062 --> 00:46:51,138
autonomous trucks and
production, right? it's a thing

818
00:46:51,138 --> 00:46:54,447
that, you put together your
first toy robot and then you,

819
00:46:54,447 --> 00:46:58,046
the first thing, and you stick a
camera on it, the first thing

820
00:46:58,046 --> 00:47:01,586
you're going to want to do is
like, all right, what can I see

821
00:47:01,586 --> 00:47:04,895
the data coming out of the
camera? can I see, like, where

822
00:47:04,895 --> 00:47:08,320
this sort of robot thinks it's,
it is in its, surroundings?

823
00:47:08,320 --> 00:47:11,570
You're doing some, slam, or
you're doing a 2D LiDAR, or,

824
00:47:11,570 --> 00:47:14,821
even the most basic robot, you
immediately run into this

825
00:47:14,821 --> 00:47:18,245
problem of how do I visualize,
what the robot is seeing and

826
00:47:18,245 --> 00:47:19,000
thinking? And

827
00:47:19,000 --> 00:47:22,683
Audrow Nash: Yeah. How do I,
check where I am or where the

828
00:47:22,683 --> 00:47:23,000
robot

829
00:47:23,000 --> 00:47:26,547
Adrian Macneil: And it's an
inherently multimodal problem.

830
00:47:26,547 --> 00:47:30,461
It's not something that you can
just look at text logs or, plots

831
00:47:30,461 --> 00:47:34,069
and things to understand. you
need to be able to see across

832
00:47:34,069 --> 00:47:37,800
all of this data. You need to
see in 3d space, a model of the

833
00:47:37,800 --> 00:47:41,285
robot and what it thinks its
surroundings are. And things

834
00:47:41,285 --> 00:47:45,199
like, lidars and radars that are
giving you a sense of depth and

835
00:47:45,199 --> 00:47:48,930
space. And then you also want to
see video data coming out of

836
00:47:48,930 --> 00:47:52,721
cameras. You also want to see,
GPS data on a map if it's got a

837
00:47:52,721 --> 00:47:56,452
GPS. You want to see you do have
a lot of like plot data. You

838
00:47:56,452 --> 00:47:59,938
want to plot things like joint
sort of positions or motor

839
00:47:59,938 --> 00:48:03,546
torque velocity, things like
this. You do have text logs as

840
00:48:03,546 --> 00:48:07,399
well, but you have all of this,
right? You have plots, you have

841
00:48:07,399 --> 00:48:11,068
text logs, you have maps, you
have 3D, you have cameras, and

842
00:48:11,068 --> 00:48:14,615
so you need a tool that is
inherently a sort of multimodal

843
00:48:14,615 --> 00:48:17,000
visualizer and that's what
Voxlog does.

844
00:48:17,000 --> 00:48:21,960
Audrow Nash: yeah. Yeah. So it
makes it so you can. Understand

845
00:48:21,960 --> 00:48:26,840
the state of the robot so you
can understand what it's doing,

846
00:48:26,840 --> 00:48:31,560
so you can develop things, so
you can understand what might

847
00:48:31,560 --> 00:48:36,120
have gone wrong or see if
there's any errors or anything.

848
00:48:36,120 --> 00:48:37,000
lots of use

849
00:48:37,000 --> 00:48:41,421
Adrian Macneil: if, I, you could
say that it helps you see what

850
00:48:41,421 --> 00:48:45,000
the robot is, how the robot
sense, think, and acts.

851
00:48:45,000 --> 00:48:46,000
Audrow Nash: Yeah, very

852
00:48:46,000 --> 00:48:49,091
Adrian Macneil: it's, yeah, it's
what is the robot seeing? What

853
00:48:49,091 --> 00:48:52,132
is the robot thinking? how do we
understand, what is the robot

854
00:48:52,132 --> 00:48:55,125
thinking, which is usually what
is the robot's kind of mental

855
00:48:55,125 --> 00:48:58,215
model of the world and what are
the actions that it is planning

856
00:48:58,215 --> 00:48:59,000
on taking? Yeah.

857
00:48:59,000 --> 00:49:03,148
Audrow Nash: cool. Okay. What
are so thinking about

858
00:49:03,148 --> 00:49:08,191
visualizations? You've mentioned
several But what are like the

859
00:49:08,191 --> 00:49:13,316
big? Classes of visualizations.
I'm thinking like you have your

860
00:49:13,316 --> 00:49:18,359
plots you have your 3d One where
you have the point clouds and

861
00:49:18,359 --> 00:49:23,321
maybe your robot model something
like Rviz and maybe then you

862
00:49:23,321 --> 00:49:28,201
just have logs that say messages
that you want to introspect

863
00:49:28,201 --> 00:49:33,000
something, how do you think of
the big classes of you might

864
00:49:33,000 --> 00:49:37,122
Adrian Macneil: Yeah. Yeah. 3D,
3D and images are definitely One

865
00:49:37,122 --> 00:49:40,730
of the larger buckets, all of
these things are critical,

866
00:49:40,730 --> 00:49:44,723
right? But I would say like 3D
and images. So we have, a panel

867
00:49:44,723 --> 00:49:48,781
for visualizing 3D data. And we
have a panel for visualizing 2D

868
00:49:48,781 --> 00:49:52,646
images. They're actually the
same panel under the hood, this

869
00:49:52,646 --> 00:49:56,511
is a fun fact. So the 3D, we
have a panel for, the, 3D scene

870
00:49:56,511 --> 00:50:00,182
can render anything from a 3D
point cloud, a 3D, bounding

871
00:50:00,182 --> 00:50:03,918
boxes of objects, like you said,
a 3D robot model, and the

872
00:50:03,918 --> 00:50:07,783
robot's pose, so like where it's
holding its arms or joints,

873
00:50:07,783 --> 00:50:11,970
things like this, also in the 3D
panel, you can render a 2D image

874
00:50:11,970 --> 00:50:15,835
into that scene, right? So if
you've got a camera mounted on

875
00:50:15,835 --> 00:50:20,021
the robot, you can project that,
that camera image into, 3D space

876
00:50:20,021 --> 00:50:24,015
to see, what it is seeing from
that perspective. conversely in

877
00:50:24,015 --> 00:50:28,073
the image panel, you can start
with a layer of what is an image

878
00:50:28,073 --> 00:50:31,873
that we're seeing. So this is
just like 2D pixel coordinate

879
00:50:31,873 --> 00:50:35,996
space. but you can project into
that image anything from your 3D

880
00:50:35,996 --> 00:50:39,603
scene as well. So you can
project Over it, point clouds.

881
00:50:39,603 --> 00:50:43,081
As long as you've got
transforms, right? As long as we

882
00:50:43,081 --> 00:50:46,624
understand, the frame of of
camera and the relationship

883
00:50:46,624 --> 00:50:50,424
between your camera and your
LiDAR or things like this, but

884
00:50:50,424 --> 00:50:54,289
we can project over the camera,
point clouds. We can project

885
00:50:54,289 --> 00:50:58,412
over the camera bounding boxes.
So you can actually see anything

886
00:50:58,412 --> 00:51:02,276
from the 3D scene in the 2D
scene, you can see anything from

887
00:51:02,276 --> 00:51:06,141
the 2D scene in the 3D scene,
and the, even though the, the,

888
00:51:06,141 --> 00:51:09,748
UI for them is slightly
different, they're actually like

889
00:51:09,748 --> 00:51:13,871
the same piece of code under the
hood. so that's one. really big

890
00:51:13,871 --> 00:51:17,607
bucket, I would say. And then
the other, main, or big, one

891
00:51:17,607 --> 00:51:21,536
that's just like heavy uses of
plots. so plots are just, very

892
00:51:21,536 --> 00:51:25,401
important, obviously for time
series data. lots of different

893
00:51:25,401 --> 00:51:29,072
sort of types of things that
people are trying to look at

894
00:51:29,072 --> 00:51:32,873
there. But, yeah. this can be
things like encoders, motors,

895
00:51:32,873 --> 00:51:36,802
joints, there's, any number of
any, sort of numeric, any time

896
00:51:36,802 --> 00:51:40,667
series data that you've got, you
can plot there. or not time

897
00:51:40,667 --> 00:51:44,596
series, you want to plot things
like, planned velocity in the

898
00:51:44,596 --> 00:51:48,203
future, or, you want to plot
sort of movement even in 2D

899
00:51:48,203 --> 00:51:52,325
space, all sorts of things that
you can do with plots. And then,

900
00:51:52,325 --> 00:51:54,000
yeah, other ones that get.

901
00:51:54,000 --> 00:51:57,447
Adrian Macneil: but there's, we
have about 20 different types of

902
00:51:57,447 --> 00:52:00,840
visualizations that you can do
in Foxglove and an extension API

903
00:52:00,840 --> 00:52:03,964
that you can create your own
visualization panels. but the

904
00:52:03,964 --> 00:52:07,034
raw message run, think you
called out, so just being able

905
00:52:07,034 --> 00:52:10,427
to inspect a specific, again, in
the ROS world, you can do this

906
00:52:10,427 --> 00:52:13,820
by opening yet another terminal
window and doing rostopic echo.

907
00:52:13,820 --> 00:52:17,214
you still got, look at like an
average ROS developer workspace.

908
00:52:17,214 --> 00:52:20,284
There's always like five
terminal windows and like three,

909
00:52:20,284 --> 00:52:23,623
and Rviz and RQT and, there's
just like 18 windows rearranged.

910
00:52:23,623 --> 00:52:24,000
we want

911
00:52:24,000 --> 00:52:26,000
Audrow Nash: they all have their

912
00:52:26,000 --> 00:52:30,184
Adrian Macneil: we want we want
to eliminate this. yeah, so we

913
00:52:30,184 --> 00:52:34,436
have the, what we call the, raw
message panel. That's basically

914
00:52:34,436 --> 00:52:38,687
like a ROS topic echo. It just
lets you see, inspect individual

915
00:52:38,687 --> 00:52:42,601
images and pause them. we have
an out, like a, GPS map and

916
00:52:42,601 --> 00:52:46,110
outdoor map that gives you
that's a one that's quite

917
00:52:46,110 --> 00:52:48,000
helpful if you've, if you're

918
00:52:48,000 --> 00:52:49,000
Audrow Nash: Or outdoor.

919
00:52:49,000 --> 00:52:52,883
Adrian Macneil: for outdoor
types of robots, you just want

920
00:52:52,883 --> 00:52:56,968
to, to display that on the map
and, sometimes there might not

921
00:52:56,968 --> 00:53:01,186
be coordinate mapping from the
GPS data to your 3D data, so you

922
00:53:01,186 --> 00:53:05,203
can't necessarily, you can't
necessarily just like drop in a

923
00:53:05,203 --> 00:53:09,488
street map or a, or like an open
street map type thing into your

924
00:53:09,488 --> 00:53:13,840
3D panel, but you do want to see
where the robot is in space. and

925
00:53:13,840 --> 00:53:17,857
yeah, bunch of other things like
that. Diagnostics, a lot of

926
00:53:17,857 --> 00:53:20,000
these kind of things. Text logs.

927
00:53:20,000 --> 00:53:25,821
Audrow Nash: Hell yeah. Do you,
is there any, does Foxglove,

928
00:53:25,821 --> 00:53:31,254
visualization, does it fully
replace Rviz and RQT or are

929
00:53:31,254 --> 00:53:33,000
there any missing?

930
00:53:33,000 --> 00:53:36,775
Adrian Macneil: yeah, great
question. I would say, it's

931
00:53:36,775 --> 00:53:40,892
probably impossible to fully
replace Rviz for 100 percent of

932
00:53:40,892 --> 00:53:44,941
people's workflows in that some
people, I guess a couple of

933
00:53:44,941 --> 00:53:49,333
buckets of things I'll call out.
One is people that have created

934
00:53:49,333 --> 00:53:53,382
custom Rviz plugins. we don't
have any way to Automatically

935
00:53:53,382 --> 00:53:54,000
import or

936
00:53:54,000 --> 00:53:55,000
Audrow Nash: under everything
and guys are

937
00:53:55,000 --> 00:53:58,818
Adrian Macneil: QT codebase and
we've got a web stack. We do

938
00:53:58,818 --> 00:54:02,318
have extension APIs, extension
concept. You can publish

939
00:54:02,318 --> 00:54:05,882
extensions either publicly or
internally to your sort of

940
00:54:05,882 --> 00:54:09,000
Foxglove organization. but
you're not going to be

941
00:54:09,000 --> 00:54:14,718
Audrow Nash: and with no custom
plugins on Rviz. it a fair

942
00:54:14,718 --> 00:54:16,000
parity or are

943
00:54:16,000 --> 00:54:19,547
Adrian Macneil: I would say if
running no plugins and others,

944
00:54:19,547 --> 00:54:23,151
it's pretty good. The other one
besides custom plugins I would

945
00:54:23,151 --> 00:54:26,000
pull up is, is, like the Moveit,
and I think Nav2

946
00:54:26,000 --> 00:54:27,000
Audrow Nash: all that
intergations

947
00:54:27,000 --> 00:54:30,717
Adrian Macneil: yeah, we have
some of those features, but

948
00:54:30,717 --> 00:54:34,761
there's definitely a handful of
things that, I would say we're

949
00:54:34,761 --> 00:54:38,478
not at parity, with Rviz, plus
like the MoveIt or the Nav

950
00:54:38,478 --> 00:54:39,000
plugins,

951
00:54:39,000 --> 00:54:43,000
Audrow Nash: Cause they've just
been developed for Rviz or

952
00:54:43,000 --> 00:54:44,000
Adrian Macneil: right, exactly,

953
00:54:44,000 --> 00:54:49,096
Audrow Nash: that makes Yeah.
that's, those projects are huge.

954
00:54:49,096 --> 00:54:50,000
so be a lot

955
00:54:50,000 --> 00:54:53,169
Adrian Macneil: And things that
we would definitely like to,

956
00:54:53,169 --> 00:54:56,338
had, plenty of chats with those
teams and the things that we

957
00:54:56,338 --> 00:54:59,560
would like to support in the
future and feature requests come

958
00:54:59,560 --> 00:55:02,835
up reasonably frequently, but
like you say, it's just, there's

959
00:55:02,835 --> 00:55:05,000
a lot of, surface area and
visualization.

960
00:55:05,000 --> 00:55:09,933
Audrow Nash: very because it
turns into a browser or

961
00:55:09,933 --> 00:55:15,530
something. it is a browser, but
it's like it, the amount of

962
00:55:15,530 --> 00:55:21,221
visualization and display and
eventually maybe you want a UI

963
00:55:21,221 --> 00:55:26,818
all sorts of things. It just, it
becomes like a sandbox for

964
00:55:26,818 --> 00:55:29,000
making other is what it

965
00:55:29,000 --> 00:55:32,483
Adrian Macneil: yeah, it's,
it's, it is very open ended, and

966
00:55:32,483 --> 00:55:35,967
Our goal is not to solve 100
percent of anyone's we to solve

967
00:55:35,967 --> 00:55:39,741
want 90 percent of the problems,
but every, company is different.

968
00:55:39,741 --> 00:55:42,992
There's different types of
visualizations You need to be

969
00:55:42,992 --> 00:55:46,534
extensible and let people, get
the last 10 percent of the way

970
00:55:46,534 --> 00:55:49,901
themselves, on the flip side,
not have to start from zero.

971
00:55:49,901 --> 00:55:53,210
There's a lot of companies,
especially outside of the ROS

972
00:55:53,210 --> 00:55:56,810
ecosystem, a lot of companies
that have just, that are using a

973
00:55:56,810 --> 00:56:00,293
completely in house visualizer
that They have built over the

974
00:56:00,293 --> 00:56:03,835
years, right? And again, this is
something that, maybe that's

975
00:56:03,835 --> 00:56:06,796
taking up like half an
engineer's time or something

976
00:56:06,796 --> 00:56:10,337
within the company or depends on
the company size. Maybe it's

977
00:56:10,337 --> 00:56:13,937
someone contributing every, once
a week, and maybe it's like a

978
00:56:13,937 --> 00:56:17,537
full time team of a couple of
people working on it, but it's a

979
00:56:17,537 --> 00:56:20,497
significant investment in, 90
percent of that being

980
00:56:20,497 --> 00:56:23,923
undifferentiated, right? 90
percent of that being the exact

981
00:56:23,923 --> 00:56:27,000
same shit that like every
robotics company visualize.

982
00:56:27,000 --> 00:56:30,089
Audrow Nash: yep, for sure. And
also I'm sure that their in

983
00:56:30,089 --> 00:56:33,387
house one would not be as good
as something that is, that's the

984
00:56:33,387 --> 00:56:36,581
only thing you got. you guys
have several parts this project,

985
00:56:36,581 --> 00:56:37,000
but it's

986
00:56:37,000 --> 00:56:39,000
Adrian Macneil: our is yeah, we
got the whole,

987
00:56:39,000 --> 00:56:41,000
Audrow Nash: a robotics company.

988
00:56:41,000 --> 00:56:44,385
Adrian Macneil: exactly.
leverage a team of, we got ten

989
00:56:44,385 --> 00:56:48,015
engineers right now, I think,
focusing on, leverage a whole

990
00:56:48,015 --> 00:56:49,000
team focused on,

991
00:56:49,000 --> 00:56:50,000
Audrow Nash: for sure.

992
00:56:50,000 --> 00:56:53,231
Audrow Nash: And you guys have
been working on it for years and

993
00:56:53,231 --> 00:56:56,000
out of Cruise, which I don't
think we said explicitly,

994
00:56:56,000 --> 00:56:59,563
Adrian Macneil: yeah, and one
of, a couple of our engineers,

995
00:56:59,563 --> 00:57:02,889
have been working on this
project for seven, eight years

996
00:57:02,889 --> 00:57:06,452
now. It's yeah, the visualizer
did actually, start at Cruise

997
00:57:06,452 --> 00:57:09,956
and some of our, very small
portion of our code traces its,

998
00:57:09,956 --> 00:57:13,460
traces its way back that because
created this visualizer at

999
00:57:13,460 --> 00:57:16,786
Cruise, we partially open
sourced it at Cruise, it got a

1000
00:57:16,786 --> 00:57:20,528
little bit of traction, but it
was also, too tightly coupled to

1001
00:57:20,528 --> 00:57:24,269
a bunch Cruise assumptions and
self driving assumptions and was

1002
00:57:24,269 --> 00:57:27,832
not as useful for other types of
robots, other types of like

1003
00:57:27,832 --> 00:57:31,099
manipulators or mobile
manipulators or AMRs and things.

1004
00:57:31,099 --> 00:57:34,721
it was, like, had a lot of built
in assumptions around you're

1005
00:57:34,721 --> 00:57:38,344
probably going to be outdoor,
you're probably not going to be

1006
00:57:38,344 --> 00:57:41,848
able to move the camera in
certain ways, your robot, a self

1007
00:57:41,848 --> 00:57:45,293
driving car doesn't have any
sort of or, really like joint

1008
00:57:45,293 --> 00:57:48,737
states or anything like this. so
it was, we partially open

1009
00:57:48,737 --> 00:57:52,063
sourced it. at Cruise and it did
get some adoption, with

1010
00:57:52,063 --> 00:57:55,626
Foxglove, it was like, hey, all
right, let's, take that as a

1011
00:57:55,626 --> 00:57:59,249
foundation, move it on. I think
now we're, we look at it's 15

1012
00:57:59,249 --> 00:58:02,456
percent or something of our
code. It dates back to the

1013
00:58:02,456 --> 00:58:04,000
Cruise days. It's a pretty

1014
00:58:04,000 --> 00:58:06,000
Audrow Nash: Yeah. And it's

1015
00:58:06,000 --> 00:58:09,850
Adrian Macneil: yeah, small and
decreasing fraction, but, but it

1016
00:58:09,850 --> 00:58:13,519
is that lineage of the project
that we've been working on and

1017
00:58:13,519 --> 00:58:14,000
some our

1018
00:58:14,000 --> 00:58:15,000
Audrow Nash: For sure.

1019
00:58:15,000 --> 00:58:16,000
Adrian Macneil: time now.

1020
00:58:16,000 --> 00:58:17,000
Audrow Nash: Hell yeah.

1021
00:58:17,000 --> 00:58:21,000
Audrow Nash: Now tell me about
the plugins and extension API.

1022
00:58:21,000 --> 00:58:24,319
Adrian Macneil: Yeah. So we
have, Bunch of different ways

1023
00:58:24,319 --> 00:58:27,521
that you can customize, the
visualizations. Like I say,

1024
00:58:27,521 --> 00:58:30,898
everyone, there's a lot of
different things people want to

1025
00:58:30,898 --> 00:58:34,565
do with visualizations. So at a
first kind of premise and, also

1026
00:58:34,565 --> 00:58:38,292
in addition to that, one of our
kind of founding beliefs is that

1027
00:58:38,292 --> 00:58:40,912
robotics tooling and
infrastructure should be

1028
00:58:40,912 --> 00:58:44,230
framework agnostic. So quite
early on, quite early on, we

1029
00:58:44,230 --> 00:58:47,782
started with ROS support, but
quite early on we, we tried to,

1030
00:58:47,782 --> 00:58:51,275
as much as possible, Decouple
ourselves from ROS while still

1031
00:58:51,275 --> 00:58:54,826
supporting ROS really well and
make it in a way that anything

1032
00:58:54,826 --> 00:58:58,145
that you can do in Foxglove,
nothing in Foxglove requires

1033
00:58:58,145 --> 00:59:01,755
ROS. Anything can be done just
in a general purpose way. so we

1034
00:59:01,755 --> 00:59:05,423
think about extensibility. The
first thing doesn't even require

1035
00:59:05,423 --> 00:59:08,858
an extension. It's just how do
you render arbitrary things,

1036
00:59:08,858 --> 00:59:12,351
right? So we have a bunch of
primitives that you can render.

1037
00:59:12,351 --> 00:59:15,902
You can publish messages, either
directly from your robot, or

1038
00:59:15,902 --> 00:59:18,988
with a, what we call a user
script, which is a little

1039
00:59:18,988 --> 00:59:22,714
script. Running, in line in the
browser that you write that does

1040
00:59:22,714 --> 00:59:26,149
transformations. but you can
take your, can take your robot

1041
00:59:26,149 --> 00:59:29,351
data and you can turn that into
things like we have, 3D

1042
00:59:29,351 --> 00:59:32,612
primitives, like a cube
primitive or a line primitive or

1043
00:59:32,612 --> 00:59:36,396
a, point primitive or a. A Model
Primitive, if you want to render

1044
00:59:36,396 --> 00:59:40,064
a mesh or something like that.
so we have these building blocks

1045
00:59:40,064 --> 00:59:43,557
of you want to, sure, you want a
stop sign in your 3D panel.

1046
00:59:43,557 --> 00:59:46,992
Okay, just bring a mesh, bring
an STL file or, or something

1047
00:59:46,992 --> 00:59:50,602
like that. And you can publish
that, wrap it up inside a Model

1048
00:59:50,602 --> 00:59:54,095
Primitive message and publish
that, to Foxglove. So you have

1049
00:59:54,095 --> 00:59:57,181
first of all, these building
blocks of like, how do I

1050
00:59:57,181 --> 01:00:00,733
visualize things, As well as
more common sort of higher level

1051
01:00:00,733 --> 01:00:04,459
abstractions like a point cloud,
for example, great, we can just

1052
01:00:04,459 --> 01:00:08,185
take a point cloud message, and
visualize that you don't have to

1053
01:00:08,185 --> 01:00:09,000
write out, 30,

1054
01:00:09,000 --> 01:00:11,000
Audrow Nash: Yeah. already
there.

1055
01:00:11,000 --> 01:00:14,152
Adrian Macneil: or anything
crazy like that. and then, so

1056
01:00:14,152 --> 01:00:17,304
you have all of these kind of
like drawing primitives and

1057
01:00:17,304 --> 01:00:20,622
that's the, that's the first
place that you would start. And

1058
01:00:20,622 --> 01:00:23,000
then, like I said, you get into
extensions.

1059
01:00:23,000 --> 01:00:26,557
Adrian Macneil: We have, a thing
called user scripts that I

1060
01:00:26,557 --> 01:00:29,873
mentioned before where you
Basically subscribe to topic

1061
01:00:29,873 --> 01:00:33,671
data coming from your robot that
might be a custom message that

1062
01:00:33,671 --> 01:00:37,590
you've written. Let's say you've
got a custom message that's like

1063
01:00:37,590 --> 01:00:41,207
tracked objects or something,
right? You want to render that

1064
01:00:41,207 --> 01:00:44,945
so you can write a user script
that just says read the tracked

1065
01:00:44,945 --> 01:00:48,864
object and output, like an array
of, like cubes or something like

1066
01:00:48,864 --> 01:00:52,541
this cube primitives. so that's,
just like a really quick way

1067
01:00:52,541 --> 01:00:56,400
that you can turn arbitrary data
into something visual. And then

1068
01:00:56,400 --> 01:01:00,138
we have extension versions of
that. So the user script is like

1069
01:01:00,138 --> 01:01:03,514
a lightweight, just like
happening fully in the browser.

1070
01:01:03,514 --> 01:01:07,191
You're editing it and there's
like a full on text editor just

1071
01:01:07,191 --> 01:01:09,000
sitting there ready to go. is,

1072
01:01:09,000 --> 01:01:12,000
Audrow Nash: nice. That's so

1073
01:01:12,000 --> 01:01:14,291
Adrian Macneil: yeah, It it,
it's just a lot easier to play

1074
01:01:14,291 --> 01:01:16,000
around with things basically.
You don't have

1075
01:01:16,000 --> 01:01:19,576
Audrow Nash: do something very
quick. And if you have a very

1076
01:01:19,576 --> 01:01:23,272
tiny. Transform that you want to
apply to some data so you can

1077
01:01:23,272 --> 01:01:25,000
display it. That's very Okay,

1078
01:01:25,000 --> 01:01:26,000
Adrian Macneil: Yeah.

1079
01:01:25,000 --> 01:01:28,702
Audrow Nash: that's for only
lightweight things, because I

1080
01:01:28,702 --> 01:01:31,000
would assume, I like my, code
editor

1081
01:01:31,000 --> 01:01:32,000
Adrian Macneil: right,

1082
01:01:32,000 --> 01:01:35,000
Audrow Nash: So then if you have
larger projects, now you

1083
01:01:35,000 --> 01:01:38,508
Adrian Macneil: Yeah. So that's
when you would leap to like. A

1084
01:01:38,508 --> 01:01:41,847
full on what we call extension,
with extensions, there's, a

1085
01:01:41,847 --> 01:01:45,412
handful of different extension
types. You can, you can do those

1086
01:01:45,412 --> 01:01:48,637
kind of transforms. So for
example, we have, I think it's

1087
01:01:48,637 --> 01:01:51,975
called a message converter
extension where you can, you can

1088
01:01:51,975 --> 01:01:55,597
say, anytime we see a topic, of
type foo, here is a transform, a

1089
01:01:55,597 --> 01:01:59,275
known transform that would apply
that and turn it into a Foxglove

1090
01:01:59,275 --> 01:02:02,500
schema, something that we know
how to visualize. So maybe

1091
01:02:02,500 --> 01:02:06,065
you've mucked around with your
user script and from you want to

1092
01:02:06,065 --> 01:02:09,460
go from your tracked objects to
like something visual You're

1093
01:02:09,460 --> 01:02:13,081
like, okay. that's nice Audrow's
figured this out How do we, but

1094
01:02:13,081 --> 01:02:16,420
I've got a team of 50 engineers
who all want to do the same

1095
01:02:16,420 --> 01:02:19,815
thing You can bring that into
your code editor, bundle up as

1096
01:02:19,815 --> 01:02:23,266
an extension, deploy it to your
company as this topic matches

1097
01:02:23,266 --> 01:02:26,718
this like particular internal
data type This is how you would

1098
01:02:26,718 --> 01:02:30,000
turn it into something that
FoxLog knows how to visualize.

1099
01:02:30,000 --> 01:02:34,233
Audrow Nash: So with that,
anytime the topic matches, do

1100
01:02:34,233 --> 01:02:38,767
you, so I, do you have, do you
code it in the extension that

1101
01:02:38,767 --> 01:02:43,000
I'm looking for a topic named
foobar joints or whatever?

1102
01:02:43,000 --> 01:02:45,000
Adrian Macneil: or with with
that

1103
01:02:45,000 --> 01:02:46,000
Audrow Nash: is that

1104
01:02:46,000 --> 01:02:49,529
Adrian Macneil: You with that
one in the extension With that

1105
01:02:49,529 --> 01:02:53,000
example, in the extension, you
match on a type, like a data

1106
01:02:53,000 --> 01:02:56,000
Audrow Nash: Ah, cool. Way
better.

1107
01:02:56,000 --> 01:02:59,676
Adrian Macneil: slash, tracked
objects or something, that's

1108
01:02:59,676 --> 01:03:03,290
kind of custom data type that
you've created, so you would

1109
01:03:03,290 --> 01:03:07,215
just match on that, it doesn't
necessarily need to be the topic

1110
01:03:07,215 --> 01:03:10,704
name, it's just the, any,
message that matches type, and

1111
01:03:10,704 --> 01:03:14,069
then how and then this is
something, again, you, would

1112
01:03:14,069 --> 01:03:16,000
write this in your code editor.

1113
01:03:16,000 --> 01:03:17,000
Audrow Nash: Yeah.

1114
01:03:17,000 --> 01:03:23,240
Adrian Macneil: Yeah. We have
both. are topic based and the,

1115
01:03:23,240 --> 01:03:28,960
extension is a, yeah, the
extension ones work on a type

1116
01:03:28,960 --> 01:03:30,000
base, but,

1117
01:03:30,000 --> 01:03:34,785
Audrow Nash: Cool. Okay. Yeah,
okay, I was just thinking how to

1118
01:03:34,785 --> 01:03:36,000
make it not as A

1119
01:03:36,000 --> 01:03:38,000
Adrian Macneil: Yeah, You don't
to be

1120
01:03:38,000 --> 01:03:41,480
Audrow Nash: and maybe I'm
coming with Ross, two ideas. So

1121
01:03:41,480 --> 01:03:44,000
maybe the, language is slightly
different.

1122
01:03:44,000 --> 01:03:47,283
Adrian Macneil: Yeah, but the
thing, yeah, the way we think

1123
01:03:47,283 --> 01:03:50,398
about the type thing is, it's,
really just like teaching

1124
01:03:50,398 --> 01:03:53,792
Foxglove how to visualize your
custom message, and it doesn't

1125
01:03:53,792 --> 01:03:57,131
matter what topic it's coming
on, it's just anytime you see,

1126
01:03:57,131 --> 01:04:00,636
because we know how to visualize
it. A ROS point cloud. We know

1127
01:04:00,636 --> 01:04:04,141
how to visualize, or we know how
to visualize like a ROS marker

1128
01:04:04,141 --> 01:04:07,480
message or something. We don't
know how to visualize like an

1129
01:04:07,480 --> 01:04:10,595
Audrow tracked object or
whatever, so we create, you can

1130
01:04:10,595 --> 01:04:14,045
teach us how to visualize that
by creating one of these little

1131
01:04:14,045 --> 01:04:17,383
like transformer extensions,
converter extensions, and then,

1132
01:04:17,383 --> 01:04:20,777
and then you can deploy that,
like I say, either publicly or.

1133
01:04:20,777 --> 01:04:21,000
Just

1134
01:04:21,000 --> 01:04:23,000
Audrow Nash: Internally or

1135
01:04:23,000 --> 01:04:24,000
Adrian Macneil: yeah.

1136
01:04:24,000 --> 01:04:27,000
Audrow Nash: That's really cool.
yeah. Hell

1137
01:04:27,000 --> 01:04:29,921
Adrian Macneil: and then
there's, there's three or four

1138
01:04:29,921 --> 01:04:33,161
different extension types, but
the other kind of key one that

1139
01:04:33,161 --> 01:04:36,242
people, make a lot of use of is
you can actually create an

1140
01:04:36,242 --> 01:04:39,588
entirely custom panel, right? So
each panel in Foxglove is just

1141
01:04:39,588 --> 01:04:42,828
its own separate little HTML
JavaScript app. and the simplest

1142
01:04:42,828 --> 01:04:46,068
possible panel you could create
would just be, exactly, Hello

1143
01:04:46,068 --> 01:04:49,043
World, or just a button or
something, right? You can, in

1144
01:04:49,043 --> 01:04:51,964
fact, the button example is
quite cool, because you can

1145
01:04:51,964 --> 01:04:55,257
just, you can use the API, you
can create a little, just, HTML

1146
01:04:55,257 --> 01:04:58,604
page with a button on subscribe
to any button clicks, you could

1147
01:04:58,604 --> 01:05:01,897
actually publish a message back
to the robot, for example. and

1148
01:05:01,897 --> 01:05:05,084
is a this cool way, we have a
built in kind of button panel,

1149
01:05:05,084 --> 01:05:08,377
but it's people want a lot of
different things out of buttons,

1150
01:05:08,377 --> 01:05:11,192
so you may as well just make
your little form, create

1151
01:05:11,192 --> 01:05:13,954
whatever, you can display
whatever you want, you can

1152
01:05:13,954 --> 01:05:17,141
handle button clicks, whatever
you want, and you can publish

1153
01:05:17,141 --> 01:05:19,000
messages back to the robot as
well.

1154
01:05:19,000 --> 01:05:24,106
Audrow Nash: Now given. Like I,
I've worked with, like I, I

1155
01:05:24,106 --> 01:05:28,000
tried to make some Foxglove
extensions in the

1156
01:05:28,000 --> 01:05:29,000
Adrian Macneil: Oh yeah, cool.

1157
01:05:29,000 --> 01:05:33,118
Audrow Nash: and, but, so with
this, it's all, so you're

1158
01:05:33,118 --> 01:05:37,750
running Electron and your UI is,
it's a type script with React,

1159
01:05:37,750 --> 01:05:42,015
right? if you wanna add these,
ru these buttons and things

1160
01:05:42,015 --> 01:05:44,000
you're using TypeScript and

1161
01:05:44,000 --> 01:05:47,531
Adrian Macneil: don't, there's
no, we, use TypeScript and

1162
01:05:47,531 --> 01:05:51,000
React, there's no requirement,
on that for an extension.

1163
01:05:51,000 --> 01:05:54,938
Audrow Nash: the, if you wanted
to do your custom one, is there

1164
01:05:54,938 --> 01:05:55,000
a

1165
01:05:55,000 --> 01:05:59,073
Adrian Macneil: No, you can, we
do, I think. because we use

1166
01:05:59,073 --> 01:06:03,215
TypeScript, I think we, a lot
like we generate and we'll use

1167
01:06:03,215 --> 01:06:07,149
But there's no, TypeScript is
just compiled to JavaScript

1168
01:06:07,149 --> 01:06:11,291
under the hood. so you can use
plain JavaScript if you want.

1169
01:06:11,291 --> 01:06:15,571
and then, but as far as React,
there's no, there's actually no

1170
01:06:15,571 --> 01:06:19,575
dependency on React. Within the
panels themselves. So each

1171
01:06:19,575 --> 01:06:23,716
panel, if right, you want to add
React, do. That's fine. But

1172
01:06:23,716 --> 01:06:27,996
yeah, you can just stick like a
very basic, hello world, HTML,

1173
01:06:27,996 --> 01:06:32,000
like you would have written in
of thing there if you want.

1174
01:06:32,000 --> 01:06:35,241
Audrow Nash: it's very flexible.
You basically say load this web

1175
01:06:35,241 --> 01:06:38,380
page from or something like this
and it just does it. Okay, so

1176
01:06:38,380 --> 01:06:40,000
you get all that flexibility in.

1177
01:06:40,000 --> 01:06:43,571
Adrian Macneil: And then, yeah.
then up to you if you want to.

1178
01:06:43,571 --> 01:06:47,141
I've seen some people do crazy
things with extensions. You can

1179
01:06:47,141 --> 01:06:50,309
use that to load some web
assembly or, do, anything you

1180
01:06:50,309 --> 01:06:51,000
want really.

1181
01:06:51,000 --> 01:06:56,294
Audrow Nash: Oh, that's really
cool. Yeah. WebAssembly is so

1182
01:06:56,294 --> 01:06:57,000
exciting

1183
01:06:57,000 --> 01:07:00,621
Adrian Macneil: Yeah, some very,
yeah, it's still, there's very

1184
01:07:00,621 --> 01:07:04,126
interesting things being done
with WebAssembly, it's still an

1185
01:07:04,126 --> 01:07:07,345
earlier stage technology and
there's a lot of, there are

1186
01:07:07,345 --> 01:07:10,448
rough edges around how you move
back and forth between

1187
01:07:10,448 --> 01:07:13,954
WebAssembly things and browser
things, but it's, coming along

1188
01:07:13,954 --> 01:07:17,575
really nicely and we do use, we
use WebAssembly internally, for

1189
01:07:17,575 --> 01:07:21,310
Some performance critical things
too, like when you get into, for

1190
01:07:21,310 --> 01:07:24,989
example, like decoding bags and
you're doing like decompression,

1191
01:07:24,989 --> 01:07:28,667
lc4 decompression, lc standard,
things like this. we're actually

1192
01:07:28,667 --> 01:07:32,000
using what you are getting in
Foxglove is C plus plus code

1193
01:07:32,000 --> 01:07:35,276
that's being compiled to
WebAssembly to do a lot of that,

1194
01:07:35,276 --> 01:07:37,000
like decompression and things.

1195
01:07:37,000 --> 01:07:41,717
Audrow Nash: So it's nice and
fast. That's great. What a cool

1196
01:07:41,717 --> 01:07:46,280
thing. Yeah, I really, I'm
excited about WASM, WebAssembly,

1197
01:07:46,280 --> 01:07:50,456
just in general. so can you,
this might be a, so we've

1198
01:07:50,456 --> 01:07:55,096
already mentioned you can make
UI, which is cool. So you can

1199
01:07:55,096 --> 01:08:00,045
make a button, it can publish a
topic, and then that can be used

1200
01:08:00,045 --> 01:08:04,608
to control your robot or do
something. Or maybe even pop up

1201
01:08:04,608 --> 01:08:09,093
different parts of the UI or do
things within the Foxglove

1202
01:08:09,093 --> 01:08:13,656
program, I would imagine. Yeah.
Would you ever imagine this

1203
01:08:13,656 --> 01:08:17,987
being like a front end for a
simulation? Like we have GZ

1204
01:08:17,987 --> 01:08:22,859
server, Gazebo server and then
we have GZ, I don't know, client

1205
01:08:22,859 --> 01:08:27,344
or whatever it is, the renderer.
could you, would you ever

1206
01:08:27,344 --> 01:08:32,061
imagine using Foxglove as a way
of displaying simulation or i

1207
01:08:32,061 --> 01:08:35,000
don't know connecting it more
other to

1208
01:08:35,000 --> 01:08:36,000
Adrian Macneil: Yeah.

1209
01:08:36,000 --> 01:08:39,353
Adrian Macneil: We do see
people, like today, people use

1210
01:08:39,353 --> 01:08:43,125
Voxelver a lot to visualize the
output of a simulation. Because

1211
01:08:43,125 --> 01:08:46,298
typically simulation, there is
the perspective of the

1212
01:08:46,298 --> 01:08:49,950
simulator, and it's, you can
think of it as Okay, here's this

1213
01:08:49,950 --> 01:08:53,602
third person view of, what the
simulator is trying to create.

1214
01:08:53,602 --> 01:08:57,194
But then that data is being
created into sort of false, it's

1215
01:08:57,194 --> 01:09:01,026
creating sensor data and that's
being fed into your robot stack.

1216
01:09:01,026 --> 01:09:04,618
You do want to visualize the
other end of it, which is like,

1217
01:09:04,618 --> 01:09:08,091
what does the robot think that
it's doing? so that is very

1218
01:09:08,091 --> 01:09:11,683
common. People like visualizing
the output of simulations is

1219
01:09:11,683 --> 01:09:13,000
very common. the, sort

1220
01:09:13,000 --> 01:09:15,000
Audrow Nash: you mean the result
of what the

1221
01:09:15,000 --> 01:09:18,978
Adrian Macneil: Yeah, exactly.
Yeah, like the recording, if the

1222
01:09:18,978 --> 01:09:22,513
robot, if a robot is driving
around in the real world or

1223
01:09:22,513 --> 01:09:26,365
whatever, moving around in the
real world, yeah, exactly what

1224
01:09:26,365 --> 01:09:30,153
others would see. So the robot,
is moving around in the real

1225
01:09:30,153 --> 01:09:33,878
world, you would, you have a
recording of what it saw. if a

1226
01:09:33,878 --> 01:09:37,603
robot is, moving around in
simulation, you have a recording

1227
01:09:37,603 --> 01:09:41,518
of what the robot What things it
saw, what the robot thinks it

1228
01:09:41,518 --> 01:09:45,306
was doing, so that's like you
say that, the sort of the Rviz

1229
01:09:45,306 --> 01:09:49,284
output of, yeah, the recording
and what kind of came out of the

1230
01:09:49,284 --> 01:09:52,820
simulation, so that, that is
necessary, I guess like the

1231
01:09:52,820 --> 01:09:56,734
question then becomes, could you
have one UI, which is solving

1232
01:09:56,734 --> 01:10:00,775
both the simulation need and the
robot need, I think it could be

1233
01:10:00,775 --> 01:10:04,816
done. I think it would be a ton
of work. we don't have any plans

1234
01:10:04,816 --> 01:10:08,478
to do that. I think that, it,
logic, it would conceptually

1235
01:10:08,478 --> 01:10:12,582
make sense that you could create
all of the buttons that exist in

1236
01:10:12,582 --> 01:10:16,433
Gazebo. You could create, you
could, create a Foxglove plugin

1237
01:10:16,433 --> 01:10:20,284
or it could be a first party
supported thing and I could just

1238
01:10:20,284 --> 01:10:23,820
talk to, like you said, the
Gazebo server. but you would

1239
01:10:23,820 --> 01:10:27,735
have to recreate all of that UI
and it would be a ton of work.

1240
01:10:27,735 --> 01:10:31,586
Which, number one, so that's
like a lot of a lot of work that

1241
01:10:31,586 --> 01:10:35,311
has to be recreated. And then
you've got this problem that,

1242
01:10:35,311 --> 01:10:38,721
are a lot of simulation,
frameworks and engines in the

1243
01:10:38,721 --> 01:10:42,509
robotics space, right? And now
you've done all of that work,

1244
01:10:42,509 --> 01:10:46,613
and now you just support Gazebo,
but you don't support, IsaacSim,

1245
01:10:46,613 --> 01:10:50,527
you don't support, any number of
other, simulators that people

1246
01:10:50,527 --> 01:10:54,000
are using. yeah, and so you just
end up in this kind of

1247
01:10:54,000 --> 01:10:57,000
Audrow Nash: you've over
committed to one

1248
01:10:57,000 --> 01:10:59,718
Adrian Macneil: We've got enough
things to do in the

1249
01:10:59,718 --> 01:11:02,906
visualization space. if I think
it was done by a, I think if,

1250
01:11:02,906 --> 01:11:06,042
maybe if that was like a third
party, a plugin or something,

1251
01:11:06,042 --> 01:11:09,178
and that someone was dedicated
to maintaining that, it could

1252
01:11:09,178 --> 01:11:12,000
maybe be done. But it's, yeah,
it sounds like a lot of

1253
01:11:12,000 --> 01:11:15,792
Audrow Nash: it's not directly
interesting Alright, it's an,

1254
01:11:15,792 --> 01:11:19,710
it's, you're over committing a
bunch of resources to something

1255
01:11:19,710 --> 01:11:23,249
that kind of works anyways for
this, and I guess it does

1256
01:11:23,249 --> 01:11:27,167
require a lot of, I don't know
about the build story of Gazebo

1257
01:11:27,167 --> 01:11:29,000
on different platforms, okay,

1258
01:11:29,000 --> 01:11:32,974
Adrian Macneil: you come down to
you've done all that work and

1259
01:11:32,974 --> 01:11:36,628
now you only support one
simulator and then, other people

1260
01:11:36,628 --> 01:11:39,000
in the industry are different
things.

1261
01:11:39,000 --> 01:11:43,918
Audrow Nash: yeah, but Gazebo,
for example, it, you can swap

1262
01:11:43,918 --> 01:11:49,000
out the physics engine, and that
could still be, A cool thing.

1263
01:11:49,000 --> 01:11:53,063
Adrian Macneil: yeah, it would
be interesting and I think that

1264
01:11:53,063 --> 01:11:57,061
statement would make more sense
in the ROS world, right? Like

1265
01:11:57,061 --> 01:12:00,797
within the ROS ecosystem, it
might make more sense to, to

1266
01:12:00,797 --> 01:12:04,533
have Rviz and, Gazebo as a
single UI because that is part

1267
01:12:04,533 --> 01:12:08,465
of just one ecosystem and you
don't need to, but for us with

1268
01:12:08,465 --> 01:12:12,529
our sort of our stance on being
framework agnostic and things,

1269
01:12:12,529 --> 01:12:16,461
it's I don't want to, Have a
situation where it's we're only

1270
01:12:16,461 --> 01:12:20,000
really useful if you're using
XYZ simulator and not, a

1271
01:12:20,000 --> 01:12:23,000
Audrow Nash: Not one. Yeah.
Okay.

1272
01:12:23,000 --> 01:12:28,000
Adrian Macneil: so a little bit
of a unsolved problem there.

1273
01:12:28,000 --> 01:12:30,000
Audrow Nash: For sure.

1274
01:12:30,000 --> 01:12:33,420
Audrow Nash: is there anything
else to mention about the

1275
01:12:33,420 --> 01:12:36,840
visualizer or plugins? Or is
this the full extent of the

1276
01:12:36,840 --> 01:12:38,000
stack, or are there

1277
01:12:38,000 --> 01:12:41,348
Adrian Macneil: this is, largely
how we think about it, right? We

1278
01:12:41,348 --> 01:12:44,232
talked about recording, we
talked about getting data off

1279
01:12:44,232 --> 01:12:47,167
the robot, we talked about
indexing data in the cloud, we

1280
01:12:47,167 --> 01:12:50,052
talked about, search and
discovery of data, talked about

1281
01:12:50,052 --> 01:12:53,039
visualization. some of the
things that, you know, when you

1282
01:12:53,039 --> 01:12:56,077
think like a little bit. A
little bit down the road, things

1283
01:12:56,077 --> 01:12:58,858
that we would want to do to
round out that are around,

1284
01:12:58,858 --> 01:13:02,000
better indexing of data, better
search and query across data,

1285
01:13:02,000 --> 01:13:06,892
Audrow Nash: Yeah, I imagine
making things easier for like

1286
01:13:06,892 --> 01:13:09,000
machine learning training

1287
01:13:09,000 --> 01:13:10,000
Adrian Macneil: Yeah,

1288
01:13:09,000 --> 01:13:12,488
Audrow Nash: Because you have
your indexing that to me seems

1289
01:13:12,488 --> 01:13:16,035
like especially given like the
zeitgeist in robotics It feels

1290
01:13:16,035 --> 01:13:19,000
like everything is about All the
AI all the machine

1291
01:13:19,000 --> 01:13:20,000
Adrian Macneil: Absolutely,

1292
01:13:20,000 --> 01:13:21,000
Audrow Nash: so

1293
01:13:21,000 --> 01:13:24,818
Adrian Macneil: Yeah, and we're
still, yeah, this is yeah, it's,

1294
01:13:24,818 --> 01:13:28,577
becoming a much, a much sort of
more important part of robotics

1295
01:13:28,577 --> 01:13:32,217
development. Historically, I
think, a lot of robotics machine

1296
01:13:32,217 --> 01:13:35,916
learning was limited to just the
perception side of things and

1297
01:13:35,916 --> 01:13:39,495
just like basic image detection
models like YOLO and some of

1298
01:13:39,495 --> 01:13:43,075
these things, classification
models and things. But now, now

1299
01:13:43,075 --> 01:13:46,834
we're seeing people move to like
much more end to end learning,

1300
01:13:46,834 --> 01:13:50,533
training based on multi modal
data, not just images. and yeah,

1301
01:13:50,533 --> 01:13:54,351
like, you alluded to, you've got
all of this data in your cloud.

1302
01:13:54,351 --> 01:13:58,110
how, does that kind of feed into
your pipeline? And I think one

1303
01:13:58,110 --> 01:14:01,272
of the, one of the more
interesting pieces there that

1304
01:14:01,272 --> 01:14:05,031
we're thinking a lot about right
now is this concept of dataset

1305
01:14:05,031 --> 01:14:07,000
curation. so you've got all
these

1306
01:14:07,000 --> 01:14:12,342
Audrow Nash: don't Securation is
we said curation. set curate. Oh

1307
01:14:12,342 --> 01:14:13,000
curating

1308
01:14:13,000 --> 01:14:16,756
Adrian Macneil: yeah, set,
sorry, yeah, understands my Kiwi

1309
01:14:16,756 --> 01:14:20,576
accent, it's, yeah, the data,
yeah, so you've got, Data set,

1310
01:14:20,576 --> 01:14:24,459
right? You want to build a data
set, a training data set. and

1311
01:14:24,459 --> 01:14:28,406
you've got all of these kind of
examples. You've collected all

1312
01:14:28,406 --> 01:14:32,290
these samples of like robots
doing good things and completing

1313
01:14:32,290 --> 01:14:35,919
tasks successfully. And you've
got a bunch of examples of

1314
01:14:35,919 --> 01:14:39,738
robots doing bad things and
dropping objects or backing into

1315
01:14:39,738 --> 01:14:43,558
objects or, whatever they're
doing. you want to Add a lot of

1316
01:14:43,558 --> 01:14:47,569
metadata to these. You want to
build up a data set and you want

1317
01:14:47,569 --> 01:14:51,389
to like tag and organize them.
You want to know, do I have a

1318
01:14:51,389 --> 01:14:55,336
good distribution of data across
like different tasks that I'm

1319
01:14:55,336 --> 01:14:59,156
trying to complete? and, all of
these types of things. So we

1320
01:14:59,156 --> 01:15:02,785
need that metadata and then we
also need. full versioning

1321
01:15:02,785 --> 01:15:06,604
history of all of that metadata,
because we want to know, go

1322
01:15:06,604 --> 01:15:10,615
train a I model, I need lineage
back to what was the, the, sort

1323
01:15:10,615 --> 01:15:14,562
of data set I used to train that
model or all of the episodes,

1324
01:15:14,562 --> 01:15:18,000
all of these kind of, recordings
that we used to train

1325
01:15:18,000 --> 01:15:22,000
Audrow Nash: Yeah, you want all
of that

1326
01:15:22,000 --> 01:15:25,511
Adrian Macneil: So that becomes
interesting. then people,

1327
01:15:25,511 --> 01:15:29,022
there's, usually, you're not
taking a robot recording and

1328
01:15:29,022 --> 01:15:32,594
directly feeding that into just
like PyTorch or something,

1329
01:15:32,594 --> 01:15:36,290
right? There's, usually some
transformation that's happening

1330
01:15:36,290 --> 01:15:39,862
there. so again, these are
things that are on our mind as,

1331
01:15:39,862 --> 01:15:43,373
we think about this, life cycle
of data from, logging and

1332
01:15:43,373 --> 01:15:47,069
recording to cloud. absolutely,
things that we would like to

1333
01:15:47,069 --> 01:15:51,072
help out with in the future, but
it's also, try to make sure that

1334
01:15:51,072 --> 01:15:54,707
we do, do our core things well
before we get too distracted

1335
01:15:54,707 --> 01:15:56,000
with everything Yeah,

1336
01:15:56,000 --> 01:16:00,495
Audrow Nash: Yeah, it seems like
there's been the joke, that I've

1337
01:16:00,495 --> 01:16:04,713
seen on X quite a bit where it's
like, Everyone's digging for

1338
01:16:04,713 --> 01:16:08,793
gold. So all the AI And so
NVIDIA is selling shovels thing.

1339
01:16:08,793 --> 01:16:09,000
see

1340
01:16:09,000 --> 01:16:10,000
Adrian Macneil: doing

1341
01:16:10,000 --> 01:16:11,000
Audrow Nash: you

1342
01:16:10,000 --> 01:16:11,000
Adrian Macneil: well. Yeah.

1343
01:16:11,000 --> 01:16:15,385
Audrow Nash: but so I feel like
you guys are well positioned to

1344
01:16:15,385 --> 01:16:19,491
help with data annotation,
curation, these kinds of things.

1345
01:16:19,491 --> 01:16:23,667
cause you're already, you have
this top notch visualizer for

1346
01:16:23,667 --> 01:16:27,912
seeing data and then you could
make it so that it's very easy

1347
01:16:27,912 --> 01:16:30,000
to annotate and export to your

1348
01:16:30,000 --> 01:16:33,840
Adrian Macneil: yeah. And where
possible, there are lots of

1349
01:16:33,840 --> 01:16:37,746
existing, you think about like
image annotation or whatever,

1350
01:16:37,746 --> 01:16:41,000
right? there are lots existing,
out this. Like we,

1351
01:16:41,000 --> 01:16:43,000
Audrow Nash: Robotics

1352
01:16:43,000 --> 01:16:45,540
Adrian Macneil: yeah.
Multimodal, there are

1353
01:16:45,540 --> 01:16:49,203
definitely, fewer tools, there
are ones out there, I think, to

1354
01:16:49,203 --> 01:16:52,629
the extent possible. we would,
we can't, build everything,

1355
01:16:52,629 --> 01:16:56,173
right? We can't be the tool for
like robotics development or

1356
01:16:56,173 --> 01:16:57,000
something. the

1357
01:16:57,000 --> 01:16:58,000
Audrow Nash: got to
pragmatically

1358
01:16:58,000 --> 01:17:01,503
Adrian Macneil: my belief is
that there needs to be way more

1359
01:17:01,503 --> 01:17:04,947
companies building robotics,
developer tools, right? And we

1360
01:17:04,947 --> 01:17:08,333
would like to hey, look, we've
got the core strengths that

1361
01:17:08,333 --> 01:17:11,778
we're good at here. I'd love to
have more integrations with

1362
01:17:11,778 --> 01:17:15,105
other tools out there. And, we
talked about some of these

1363
01:17:15,105 --> 01:17:18,608
things, the, the, more of the
operational workflows and like

1364
01:17:18,608 --> 01:17:21,878
fleet management and things,
right? It's it doesn't make

1365
01:17:21,878 --> 01:17:25,556
sense for us to build everything
at once. I'd rather be good at

1366
01:17:25,556 --> 01:17:29,000
some really core, good at core,
good at some core features.

1367
01:17:29,000 --> 01:17:32,000
Audrow Nash: Yeah. I think that
makes a lot of sense.

1368
01:17:32,000 --> 01:17:35,405
Adrian Macneil: yeah, labeling,
a whole kind of can of worms. I

1369
01:17:35,405 --> 01:17:36,000
think it's,

1370
01:17:36,000 --> 01:17:39,000
Audrow Nash: it's true, a
million types of data, all the

1371
01:17:39,000 --> 01:17:42,609
Adrian Macneil: yeah, the, high
level, I, think a, much kind of,

1372
01:17:42,609 --> 01:17:45,936
more sensible thing to start
with is just more of, metadata

1373
01:17:45,936 --> 01:17:49,432
labeling, so it's like you're
going through and you're wanting

1374
01:17:49,432 --> 01:17:52,646
to, particular episodes and
which ones are good or bad or

1375
01:17:52,646 --> 01:17:55,578
which ones are matching
different criteria or, Yeah.

1376
01:17:55,578 --> 01:17:59,130
some companies are even looking
to do like, English captioning,

1377
01:17:59,130 --> 01:18:02,400
right? It's just this is a
person like picking up an apple

1378
01:18:02,400 --> 01:18:05,727
or something. because they're
feeding this into models now,

1379
01:18:05,727 --> 01:18:09,167
right? They're feeding in just
like the natural, they take an

1380
01:18:09,167 --> 01:18:12,550
existing like vision language
model off the shelf, feed in a

1381
01:18:12,550 --> 01:18:15,877
bunch of, robot data, like the
text captions plus, plus the

1382
01:18:15,877 --> 01:18:19,486
robot state and pose and things
like this, plus the sensor data,

1383
01:18:19,486 --> 01:18:23,151
feed that in and they can build,
What's called a world model that

1384
01:18:23,151 --> 01:18:26,365
has a concept of tying these
movements and things back to

1385
01:18:26,365 --> 01:18:28,000
action, which is really cool.

1386
01:18:28,000 --> 01:18:32,825
Audrow Nash: Okay. Yeah, that,
that is really cool. Are there,

1387
01:18:32,825 --> 01:18:37,650
Like we, we don't have that much
time, but, the, you'd love to

1388
01:18:37,650 --> 01:18:42,319
see way more tools in robotics.
so what would, you mentioned

1389
01:18:42,319 --> 01:18:46,988
some, but could you rattle off a
few that you think are good

1390
01:18:46,988 --> 01:18:48,000
opportunities

1391
01:18:48,000 --> 01:18:49,000
Adrian Macneil: good. Yeah.

1392
01:18:48,000 --> 01:18:51,458
Adrian Macneil: If you're going
start a robot setup today, I

1393
01:18:51,458 --> 01:18:55,031
would say if you're, thinking
this, if you're thinking about a

1394
01:18:55,031 --> 01:18:58,546
company, starting a robotics
tooling startup, just go work at

1395
01:18:58,546 --> 01:19:02,119
any robotics company for, two
years, and you'll up you'll have

1396
01:19:02,119 --> 01:19:05,404
suddenly 50 ideas for startups
for a podcast that need to

1397
01:19:05,404 --> 01:19:08,458
exist. the areas that seem
underserved, deployment to

1398
01:19:08,458 --> 01:19:11,628
robots, for example, there's
huge industry of, CICD for

1399
01:19:11,628 --> 01:19:15,085
servers, right? There's almost
no one, there was one company

1400
01:19:15,085 --> 01:19:18,428
doing this, I don't know if
they're still around. but just

1401
01:19:18,428 --> 01:19:21,540
like how do you deploy and
update robots in the field?

1402
01:19:21,540 --> 01:19:24,825
You've got to think about
deploying them without breaking

1403
01:19:24,825 --> 01:19:27,994
them. That's a, huge sort of,
thing that's interesting.

1404
01:19:27,994 --> 01:19:31,510
Configuration management for
robots out in the field. How you

1405
01:19:31,510 --> 01:19:35,198
like, keeping track of the fact
that you've got like potentially

1406
01:19:35,198 --> 01:19:38,713
different model, robot models,
different SKUs, different sort

1407
01:19:38,713 --> 01:19:42,113
of what service history has
happened. everything that every

1408
01:19:42,113 --> 01:19:45,283
company ends up building
internally should probably, if

1409
01:19:45,283 --> 01:19:48,971
it applies to multiple robotics
companies, should probably exist

1410
01:19:48,971 --> 01:19:52,141
off the shelf. So there's a lot
of the stuff around the

1411
01:19:52,141 --> 01:19:54,676
operations of just like
Deployment, Updates,

1412
01:19:54,676 --> 01:19:58,307
Configuration Management, other,
like simulation is another one

1413
01:19:58,307 --> 01:20:01,477
that I think is quite,
underserved. It's there are some

1414
01:20:01,477 --> 01:20:05,050
of these open source things like
Gazebo, but I think there's a

1415
01:20:05,050 --> 01:20:08,623
lot, there's a lot missing in
there and like frameworks to let

1416
01:20:08,623 --> 01:20:11,907
you run simulations at scale,
the frame, tools to let you

1417
01:20:11,907 --> 01:20:15,077
manage create simulation
scenarios. how, do you manage,

1418
01:20:15,077 --> 01:20:18,304
Testing a new release. you need,
like a whole catalog of

1419
01:20:18,304 --> 01:20:21,762
different scenarios that you're
ready to go simulate and you

1420
01:20:21,762 --> 01:20:25,450
need to be able to run different
aspects of that. You need to be

1421
01:20:25,450 --> 01:20:28,850
able to score the output of
simulations and say is this, is

1422
01:20:28,850 --> 01:20:32,481
this better or worse? It's not
just Hey, run this test and fail

1423
01:20:32,481 --> 01:20:36,054
if we crashed or something. It's
through the metrics, how well

1424
01:20:36,054 --> 01:20:39,684
are we doing? are we better or
worse at completing all of these

1425
01:20:39,684 --> 01:20:43,430
tasks in aggregate? There's just
endless, I could go on for hours

1426
01:20:43,430 --> 01:20:47,003
about all of these things. And.
Like I say, you just, you work

1427
01:20:47,003 --> 01:20:50,000
at any robotics company, look
around pretty quickly,

1428
01:20:50,000 --> 01:20:51,000
Audrow Nash: You

1429
01:20:51,000 --> 01:20:54,073
Adrian Macneil: you'll see
what's being built in house that

1430
01:20:54,073 --> 01:20:56,000
really is not unique to that
company.

1431
01:20:56,000 --> 01:20:59,404
Audrow Nash: Yeah. I think
that's a very good. Way to think

1432
01:20:59,404 --> 01:21:02,750
about it, just whatever's being
built in house is probably

1433
01:21:02,750 --> 01:21:06,212
something that you could hire
someone to do something better

1434
01:21:06,212 --> 01:21:09,500
and then just use. So the
doesn't have to maintain it and

1435
01:21:09,500 --> 01:21:11,000
do undifferentiating work.

1436
01:21:11,000 --> 01:21:14,307
Adrian Macneil: and the end
state of this is, quite, is,

1437
01:21:14,307 --> 01:21:17,733
that even a lot of the autonomy
that's being built is not.

1438
01:21:17,733 --> 01:21:21,513
should not be people in all our
robotics companies today I think

1439
01:21:21,513 --> 01:21:25,057
see data as a key sort of value
driver for them and they see

1440
01:21:25,057 --> 01:21:28,600
like their autonomy stack as
being like really core to their

1441
01:21:28,600 --> 01:21:32,321
mission it's okay we're solving
autonomy for but the reality is

1442
01:21:32,321 --> 01:21:35,865
that that you know when you're
creating a business the thing

1443
01:21:35,865 --> 01:21:39,586
that you are creating the value
for a company is like The value

1444
01:21:39,586 --> 01:21:42,834
that you're creating for a
customer is the problem that

1445
01:21:42,834 --> 01:21:46,496
you're solving for them and not
how you implemented a solution

1446
01:21:46,496 --> 01:21:49,862
to that problem. and a lot of
these autonomy problems are

1447
01:21:49,862 --> 01:21:53,288
like, picking and placing or
navigating around warehouses.

1448
01:21:53,288 --> 01:21:56,891
how many different Or outdoor a
lot of these kind of just and

1449
01:21:56,891 --> 01:22:00,139
that's why things like NAP
exist, right? But it's like,

1450
01:22:00,139 --> 01:22:03,742
navigating around autonomously
is not It's not a core kind of

1451
01:22:03,742 --> 01:22:07,404
value driver for your company.
What is valuable is the problem

1452
01:22:07,404 --> 01:22:10,829
that you're solving for your
customers. And quite often is

1453
01:22:10,829 --> 01:22:14,550
the distribution of how you're
getting to the customers and how

1454
01:22:14,550 --> 01:22:18,035
you're wrapping it up with
service and how and support. and

1455
01:22:18,035 --> 01:22:21,519
you think about the SaaS, the
vertical SaaS industry today.

1456
01:22:21,519 --> 01:22:24,000
There's millions of startups
building like

1457
01:22:24,000 --> 01:22:25,000
Audrow Nash: Very

1458
01:22:25,000 --> 01:22:27,596
Adrian Macneil: specific
solutions and making tons of

1459
01:22:27,596 --> 01:22:30,485
money. Tons of money, go out
there, they talk to customers,

1460
01:22:30,485 --> 01:22:33,375
they, listen to a business
problem, and then they glue five

1461
01:22:33,375 --> 01:22:36,314
things together, random open
there's, sure, there's a bit of

1462
01:22:36,314 --> 01:22:39,105
but you're not, they're not
solving a technology problem,

1463
01:22:39,105 --> 01:22:42,240
they're not taking on technology
risk, they're taking on kind of

1464
01:22:42,240 --> 01:22:44,934
product market fit risk, so they
go and, They listen to

1465
01:22:44,934 --> 01:22:47,970
customers, they build a thing as
quickly as possible by gluing

1466
01:22:47,970 --> 01:22:51,056
together as many off the shelf
components as possible, and then

1467
01:22:51,056 --> 01:22:54,191
they go and sell it, and work on
it in the market, and bundle it

1468
01:22:54,191 --> 01:22:57,227
up with support, and they solve
a problem for people, and they

1469
01:22:57,227 --> 01:22:59,970
get paid very well for doing
that. the robotics industry

1470
01:22:59,970 --> 01:23:02,957
today is the complete opposite
of that, where you have, a lot

1471
01:23:02,957 --> 01:23:05,847
of companies that are, Solving
blindingly obvious problems.

1472
01:23:05,847 --> 01:23:08,541
It's wow, if you could get a
robot to do that, would be

1473
01:23:08,541 --> 01:23:11,431
fantastic. This is that'd be
amazing. But they're taking on

1474
01:23:11,431 --> 01:23:14,565
this massive technology risk of
now I've got to get the robot to

1475
01:23:14,565 --> 01:23:17,161
do that. It's, completely
backwards. And I think when

1476
01:23:17,161 --> 01:23:20,198
robotics has succeeded, we will
see that, flip. We'll see that

1477
01:23:20,198 --> 01:23:23,185
invert, right? Where A robotic
startup will mostly be talking

1478
01:23:23,185 --> 01:23:26,369
to customers, listening to their
problem, figuring out how you're

1479
01:23:26,369 --> 01:23:29,209
going to connect with those
customers, how you're going to

1480
01:23:29,209 --> 01:23:32,148
market to them, how you're going
to sell to them, how you're

1481
01:23:32,148 --> 01:23:35,136
going to support them. And the
technology will, to the extent

1482
01:23:35,136 --> 01:23:38,221
possible, be taking together a
lot of off the shelf hardware, a

1483
01:23:38,221 --> 01:23:40,817
lot of off the shelf software, a
lot of off the shelf

1484
01:23:40,817 --> 01:23:43,707
infrastructure, a lot of off the
shelf autonomy components.

1485
01:23:43,707 --> 01:23:46,351
Probably just grabbing some off
the shelf, open source

1486
01:23:46,351 --> 01:23:49,437
foundation models. Fine tuning
them a little bit, and solving a

1487
01:23:49,437 --> 01:23:51,984
customer problem, solving a
customer need, solving a

1488
01:23:51,984 --> 01:23:55,020
business problem. Not, all of
this technology risk that people

1489
01:23:55,020 --> 01:23:56,000
are taking on today.

1490
01:23:56,000 --> 01:23:59,037
Audrow Nash: Definitely. A
hundred percent. I, this is part

1491
01:23:59,037 --> 01:24:02,228
of the reason to me, companies
like, do you know polymath They

1492
01:24:02,228 --> 01:24:03,000
were one of my,

1493
01:24:03,000 --> 01:24:05,353
Adrian Macneil: Similar kind of
thing, right? It's yeah, you

1494
01:24:05,353 --> 01:24:07,000
don't need to building the
autonomy stack.

1495
01:24:07,000 --> 01:24:11,318
Audrow Nash: yep. That's, I
think they are. early on this

1496
01:24:11,318 --> 01:24:15,939
trend and I think we're going to
see a lot more of that is my

1497
01:24:15,939 --> 01:24:20,333
guess where they're solving
navigation for those who don't

1498
01:24:20,333 --> 01:24:25,258
know and outdoor navigation like
driving slow in fields something

1499
01:24:25,258 --> 01:24:30,030
this and you just call it from
apis like that's that seems like

1500
01:24:30,030 --> 01:24:32,000
the way to go for a lot of

1501
01:24:32,000 --> 01:24:33,000
Adrian Macneil: I

1502
01:24:32,000 --> 01:24:34,000
Audrow Nash: you don't have to
do that

1503
01:24:34,000 --> 01:24:36,880
Adrian Macneil: get to a we
point where those, similar

1504
01:24:36,880 --> 01:24:40,187
companies, maybe it's Polymath,
maybe it's others, but similar

1505
01:24:40,187 --> 01:24:42,000
things exist across the board,
for

1506
01:24:42,000 --> 01:24:44,000
Audrow Nash: domain

1507
01:24:44,000 --> 01:24:47,000
Adrian Macneil: for you're
right.

1508
01:24:47,000 --> 01:24:50,000
Audrow Nash: We're on the same
page.

1509
01:24:50,000 --> 01:24:52,878
Adrian Macneil: Yeah, so I
think, yeah, we will, definitely

1510
01:24:52,878 --> 01:24:54,000
see a lot more of that.

1511
01:24:54,000 --> 01:24:55,000
Audrow Nash: hell yeah.

1512
01:24:55,000 --> 01:25:00,200
Audrow Nash: so talk to me about
the, the 2. 0 Foxglove and then,

1513
01:25:00,200 --> 01:25:05,080
the decision to make it closed
source for, I know some things

1514
01:25:05,080 --> 01:25:09,800
are open source, but, some
things are now closed So tell me

1515
01:25:09,800 --> 01:25:11,000
about all that.

1516
01:25:11,000 --> 01:25:13,864
Adrian Macneil: yeah,
absolutely. So I think there

1517
01:25:13,864 --> 01:25:17,243
were, two kind of arcs that
happened. Like I said, Foxglove

1518
01:25:17,243 --> 01:25:20,336
were about three and a half
years old now. there are a

1519
01:25:20,336 --> 01:25:24,059
couple of sort of things that we
noticed over the first few years

1520
01:25:24,059 --> 01:25:27,611
of Foxglove. the first one was
that we started out, we started

1521
01:25:27,611 --> 01:25:31,219
out with the Foxglove visualizer
And then we added on this idea

1522
01:25:31,219 --> 01:25:34,713
of we're going to build a data
platform kind of thing. And we

1523
01:25:34,713 --> 01:25:37,978
thought of them as very separate
components. And they are

1524
01:25:37,978 --> 01:25:40,842
separate components to an
extent. Like I said, the

1525
01:25:40,842 --> 01:25:44,164
visualization is useful, not
just with cloud data. You can

1526
01:25:44,164 --> 01:25:47,658
also use it with local data or
live connections. and the data

1527
01:25:47,658 --> 01:25:51,209
is independently useful. but we
took that too far and we built

1528
01:25:51,209 --> 01:25:54,245
them as Literally completely
separate code bases with

1529
01:25:54,245 --> 01:25:57,968
separate repos and UIs. and then
we start tripping over ourselves

1530
01:25:57,968 --> 01:26:01,405
with how sign on works across
them and sign out works across

1531
01:26:01,405 --> 01:26:05,071
them and how, if you're viewing
some, you're exploring some data

1532
01:26:05,071 --> 01:26:08,049
to find the right log file and
then you open it in a

1533
01:26:08,049 --> 01:26:11,314
visualizer. You've lost all
reference to how you got here

1534
01:26:11,314 --> 01:26:15,037
and what the metadata associated
with this recording was. And, if

1535
01:26:15,037 --> 01:26:18,531
you want to just be clicking
through a list of a, a list of a

1536
01:26:18,531 --> 01:26:22,082
bunch of different examples of
events or things like that, you

1537
01:26:22,082 --> 01:26:25,634
want to just be able to quickly,
Click through them. You don't

1538
01:26:25,634 --> 01:26:29,013
want a whole separate tab.
That's just the visualizer and a

1539
01:26:29,013 --> 01:26:32,450
whole separate tab That's just
the browsing interface. So we

1540
01:26:32,450 --> 01:26:35,944
came to this realization that It
was even though they are You

1541
01:26:35,944 --> 01:26:39,438
know separate product features
It was a mistake to build them

1542
01:26:39,438 --> 01:26:42,989
as completely separate apps and
completely separate UIs And so

1543
01:26:42,989 --> 01:26:46,311
we came to this decision that we
needed to, to merge these

1544
01:26:46,311 --> 01:26:49,347
together and that was the
FoxClock 2. 0 was like hey,

1545
01:26:49,347 --> 01:26:52,440
we're going from these,
completely separate, literally

1546
01:26:52,440 --> 01:26:55,590
separate code bases, separate
tools, to, to one kind of

1547
01:26:55,590 --> 01:26:59,084
observability platform, like I
said, it's modular, it's got a

1548
01:26:59,084 --> 01:27:02,406
bunch of features, you don't
have to use all of them, but,

1549
01:27:02,406 --> 01:27:06,015
it's one UI, you sign into it,
you just, it's, one kind of app,

1550
01:27:06,015 --> 01:27:09,623
and you can do things like With
1. 0, you could not do the data

1551
01:27:09,623 --> 01:27:13,117
browsing in the desktop app.
There were weird inconsistencies

1552
01:27:13,117 --> 01:27:16,440
where you could, on the web, you
could browse the data and

1553
01:27:16,440 --> 01:27:20,048
visualize it. The desktop app,
you could only visualize it, you

1554
01:27:20,048 --> 01:27:23,542
couldn't browse it. now all of
that's gone. It's one app. You

1555
01:27:23,542 --> 01:27:27,036
can use it on web, desktop. All
the features are available on

1556
01:27:27,036 --> 01:27:30,759
both platforms. and so that was,
the first key part of, 2. 0. and

1557
01:27:30,759 --> 01:27:34,139
then the second piece of it was,
like you said, around open

1558
01:27:34,139 --> 01:27:37,461
source. Our initial sort of
decision was, our initial kind

1559
01:27:37,461 --> 01:27:40,955
of like plan for the Foxglove
company was that we would build

1560
01:27:40,955 --> 01:27:44,334
this open source, visualizer,
and then we would have a paid

1561
01:27:44,334 --> 01:27:47,771
service that is like the data
management service. yeah, the,

1562
01:27:47,771 --> 01:27:48,000
kind

1563
01:27:48,000 --> 01:27:51,000
Audrow Nash: So you can get it
with

1564
01:27:51,000 --> 01:27:53,981
Adrian Macneil: yeah, with
teams. And the idea was that,

1565
01:27:53,981 --> 01:27:57,175
there would be some sort of team
visualization features, but

1566
01:27:57,175 --> 01:28:00,316
there would also be most of the
team features would revolve

1567
01:28:00,316 --> 01:28:03,297
around data management and, like
I said, uploading data,

1568
01:28:03,297 --> 01:28:05,000
organizing data in the cloud and

1569
01:28:05,000 --> 01:28:06,727
Audrow Nash: Yeah. That's what I
understood from our last

1570
01:28:06,727 --> 01:28:07,000
interview

1571
01:28:07,000 --> 01:28:10,533
Adrian Macneil: which was, that
was the plan that we set out

1572
01:28:10,533 --> 01:28:14,124
with and, we tried really hard
to make that work. I think the

1573
01:28:14,124 --> 01:28:17,657
challenges that we ran into, is
that visualization itself is

1574
01:28:17,657 --> 01:28:20,954
just a really, a bottomless, a
very difficult problem, a

1575
01:28:20,954 --> 01:28:24,545
bottomless of feature requests.
and of our effort, even, even

1576
01:28:24,545 --> 01:28:27,430
today like 80 percent of our
effort goes into the

1577
01:28:27,430 --> 01:28:30,433
visualization, right? so we
ended up with a sort of

1578
01:28:30,433 --> 01:28:34,142
challenge where we're trying to
sell a product. but some people

1579
01:28:34,142 --> 01:28:37,675
were not interested in the, some
people don't care about the

1580
01:28:37,675 --> 01:28:41,090
particular sort of the cloud
data features, right? They're

1581
01:28:41,090 --> 01:28:44,740
like, we figured out the cloud
data piece ourselves. We really

1582
01:28:44,740 --> 01:28:48,214
just want the visualization
piece. again, we try to build a

1583
01:28:48,214 --> 01:28:51,217
modular platform, so that's
nice. You just want the

1584
01:28:51,217 --> 01:28:54,749
visualization piece. but that's
a thing that we don't do. we

1585
01:28:54,749 --> 01:28:57,929
can't make money on, so it's
just an endless stream of

1586
01:28:57,929 --> 01:29:01,461
incoming feature requests and
work. and, work that we do for

1587
01:29:01,461 --> 01:29:05,053
people, but no way charge for
that. The only thing that we're

1588
01:29:05,053 --> 01:29:08,468
charging for is the, the cloud
data management, which some

1589
01:29:08,468 --> 01:29:12,295
customers, didn't want. So there
was a bit of a mismatch between,

1590
01:29:12,295 --> 01:29:15,827
what we were spending most of
our time on building, and, the

1591
01:29:15,827 --> 01:29:19,654
thing that we were charging for.
And, that, that causes a problem

1592
01:29:19,654 --> 01:29:23,246
as a company because we want to
make the visualization great.

1593
01:29:23,246 --> 01:29:26,425
But, it's just, it's not
possible to do that if you're

1594
01:29:26,425 --> 01:29:29,899
spending most of your time
building the free thing. I guess

1595
01:29:29,899 --> 01:29:33,608
another, I would say, and coming
from, I'm a developer by trade

1596
01:29:33,608 --> 01:29:37,258
and, I spend a lot of time using
a lot of open source projects

1597
01:29:37,258 --> 01:29:40,909
and we cared a lot about open
source. I think the analogy, and

1598
01:29:40,909 --> 01:29:44,265
we still care a lot about open
source, we support MCAP. I

1599
01:29:44,265 --> 01:29:47,974
think, MCAP is, an example of a
great, open source project as a

1600
01:29:47,974 --> 01:29:51,683
company, right? It's something
that you put out there. It makes

1601
01:29:51,683 --> 01:29:55,216
a lot of sense to be an open
standard. it's, a library. It's

1602
01:29:55,216 --> 01:29:58,749
not like a, it's not like an
app. So it doesn't have buttons

1603
01:29:58,749 --> 01:30:02,281
and UI and things like that.
It's a library. People are much

1604
01:30:02,281 --> 01:30:05,696
more likely to contribute to it.
We get a lot of community

1605
01:30:05,696 --> 01:30:09,052
contributions with MCAP. by
contrast, we get very, almost

1606
01:30:09,052 --> 01:30:12,231
round to zero community
contributions for a, web based

1607
01:30:12,231 --> 01:30:15,705
JavaScript visualizer tool,
right? and then also with MCAP,

1608
01:30:15,705 --> 01:30:19,238
it doesn't take a lot of our
focus, right? Maybe it's like a

1609
01:30:19,238 --> 01:30:23,006
10 percent focus for us, but not
an 80 percent focus for us. we,

1610
01:30:23,006 --> 01:30:26,656
maintain the project and keep
updates and we, there's, ongoing

1611
01:30:26,656 --> 01:30:30,366
development on it, but it's not,
It's not the, primary focus of

1612
01:30:30,366 --> 01:30:34,075
us as a company. And I think you
need to think really hard. if,

1613
01:30:34,075 --> 01:30:37,254
anyone is founding an open
source company, you need to

1614
01:30:37,254 --> 01:30:40,787
really think about like how much
the open source project is,

1615
01:30:40,787 --> 01:30:44,143
going to take of your like
overall development resources,

1616
01:30:44,143 --> 01:30:47,675
and make sure that you. that,
you have the, capacity to also

1617
01:30:47,675 --> 01:30:51,090
invest in whatever the thing
you're selling with. In some,

1618
01:30:51,090 --> 01:30:54,741
senses, and this is a little bit
of a tangent, but when people

1619
01:30:54,741 --> 01:30:58,332
ask me about creating an open
source, company, I think one of

1620
01:30:58,332 --> 01:31:02,159
the biggest challenges with open
source is that you actually have

1621
01:31:02,159 --> 01:31:05,810
two product market fit problems.
because you have a, you still

1622
01:31:05,810 --> 01:31:09,107
have, even though the open,
whatever, usually every open

1623
01:31:09,107 --> 01:31:12,816
source company has like an open
source thing and then they have

1624
01:31:12,816 --> 01:31:16,172
a paid thing that sort of
supplements it, right? and you,

1625
01:31:16,172 --> 01:31:19,587
even though the open source
thing is free, there's still a

1626
01:31:19,587 --> 01:31:23,237
product market fit problem. You
still have to make sure you're

1627
01:31:23,237 --> 01:31:26,887
creating a thing people want.
You still have to spend a lot of

1628
01:31:26,887 --> 01:31:30,656
time on kind of marketing it and
posting on social media or word

1629
01:31:30,656 --> 01:31:34,247
of mouth or whatever. You still
have to spread growth of that

1630
01:31:34,247 --> 01:31:37,662
thing. and then. That entire
thing repeats itself with the

1631
01:31:37,662 --> 01:31:41,195
paid product that you create,
right? Yeah, you have to, have

1632
01:31:41,195 --> 01:31:44,845
to create a thing people want.
It has to make sense. It has to

1633
01:31:44,845 --> 01:31:48,319
be enough of a value add, on top
of the, on top of the free

1634
01:31:48,319 --> 01:31:51,734
thing. and it's the same thing.
You have to figure out how

1635
01:31:51,734 --> 01:31:55,384
you're going to market it, how
you're going to sell it. And so

1636
01:31:55,384 --> 01:31:58,975
you've, created twice much work
for yourselves. but yeah, we,

1637
01:31:58,975 --> 01:32:02,508
just, we really found, that, we
were not able to, Invest the

1638
01:32:02,508 --> 01:32:05,923
time and effort we, needed, in
the visualization thing. We

1639
01:32:05,923 --> 01:32:09,632
weren't really getting the sort
of community contributions that

1640
01:32:09,632 --> 01:32:13,341
we would get with something like
MCAP. and frankly, a lot, most

1641
01:32:13,341 --> 01:32:17,051
of the people we talked about to
this were like, most customers

1642
01:32:17,051 --> 01:32:20,583
even, and potential customers
were like, why aren't you just

1643
01:32:20,583 --> 01:32:24,175
charging us for the visualizer?
that's the thing we want, why

1644
01:32:24,175 --> 01:32:26,000
aren't you just charging us for

1645
01:32:26,000 --> 01:32:30,000
Audrow Nash: Ah ha,

1646
01:32:30,000 --> 01:32:33,592
Adrian Macneil: but, we still
have these principles, of the,

1647
01:32:33,592 --> 01:32:37,185
that I think we're, placed when
we started, which is that we

1648
01:32:37,185 --> 01:32:40,478
want it to be, easy for
hobbyists to use. We want it to

1649
01:32:40,478 --> 01:32:44,011
be easy for academics to use.
We, we realized again, since.

1650
01:32:44,011 --> 01:32:47,603
Most of the people that were
using Foxglove, were not taking

1651
01:32:47,603 --> 01:32:51,136
advantage of the fact, they
liked the fact that there was a

1652
01:32:51,136 --> 01:32:54,549
free version, they can download
it, they can get started,

1653
01:32:54,549 --> 01:32:57,782
especially, like I said,
hobbyists, academics, or, the

1654
01:32:57,782 --> 01:33:01,255
first few people at a company
where they're just trying it

1655
01:33:01,255 --> 01:33:04,787
out, they like the fact that
it's easy to use. they weren't

1656
01:33:04,787 --> 01:33:08,559
creating a custom fork of it, or
a custom build, or deploying a

1657
01:33:08,559 --> 01:33:12,271
custom build, or anything like
that, right? It was mostly just

1658
01:33:12,271 --> 01:33:16,044
the fact that it, that there is
a free version it's easy to get

1659
01:33:16,044 --> 01:33:19,816
started with, and I think that
was well placed. So even though,

1660
01:33:19,816 --> 01:33:23,109
we, came out with this
announcement about, around 2. 0,

1661
01:33:23,109 --> 01:33:26,821
so I think it was about three
months ago, saying, hey, we, we,

1662
01:33:26,821 --> 01:33:30,593
we're not continuing any further
development on the open source

1663
01:33:30,593 --> 01:33:34,126
version of 1. x, we did have
features that were always The,

1664
01:33:34,126 --> 01:33:37,119
for example, like you go and
download the Foxglove

1665
01:33:37,119 --> 01:33:40,233
application, that always
included some closed source

1666
01:33:40,233 --> 01:33:43,885
components. So that was always
our distribution, included the

1667
01:33:43,885 --> 01:33:47,478
open source and, some of the
other components. We just said,

1668
01:33:47,478 --> 01:33:51,130
hey, the open source release,
we're not continuing to develop

1669
01:33:51,130 --> 01:33:54,782
that. the 1. x code is still
there if you need it, but we're,

1670
01:33:54,782 --> 01:33:58,614
focusing on going forward. We're
focusing on just keeping all of

1671
01:33:58,614 --> 01:34:01,967
this, within, the, closed source
build that we have been

1672
01:34:01,967 --> 01:34:05,859
releasing. honestly, it got very
little, As far as, announcements

1673
01:34:05,859 --> 01:34:09,152
go, it got pretty little
attention. And I think, like I

1674
01:34:09,152 --> 01:34:12,505
said, it didn't really change
much for most people. most

1675
01:34:12,505 --> 01:34:16,337
people were already using, they
downloaded our desktop app, that

1676
01:34:16,337 --> 01:34:19,750
was already a closed source
build because building things

1677
01:34:19,750 --> 01:34:23,642
like a macOS app with signatures
and all of these things is like,

1678
01:34:23,642 --> 01:34:27,294
it's not, particularly easy to
do anyway. That already wasn't

1679
01:34:27,294 --> 01:34:30,947
The open source build they were
using, the update to 2. 0, it

1680
01:34:30,947 --> 01:34:34,240
looks pretty similar. We've
added some new features and

1681
01:34:34,240 --> 01:34:38,072
things and we've, spent a lot of
time on performance lately, but

1682
01:34:38,072 --> 01:34:41,425
there was no disruption to
workflows, in especially, for

1683
01:34:41,425 --> 01:34:44,538
the broader community, for
individuals using it, for

1684
01:34:44,538 --> 01:34:47,771
academics using it, for
hobbyists using it. there were

1685
01:34:47,771 --> 01:34:51,424
some companies that, that were
using it in a company setting.

1686
01:34:51,424 --> 01:34:55,076
again, they were already, we
already had some of these things

1687
01:34:55,076 --> 01:34:58,788
like prompts. If if you're using
it with more than x number of

1688
01:34:58,788 --> 01:35:02,381
people, you're already seeing
hey, over three people, you've

1689
01:35:02,381 --> 01:35:06,093
got to pay. there wasn't really
any significant kind of change

1690
01:35:06,093 --> 01:35:09,686
in the experience there for
people. the only people that are

1691
01:35:09,686 --> 01:35:13,098
affected was, Sort of a small
number of people that we're

1692
01:35:13,098 --> 01:35:16,691
we're walking in. On the other
hand, I think another sort of

1693
01:35:16,691 --> 01:35:20,044
advice, piece of advice that I
give to people building a

1694
01:35:20,044 --> 01:35:23,756
similar sort of like open source
based company now, especially

1695
01:35:23,756 --> 01:35:27,588
having reflected on a few years
of this at Foxglove, is that you

1696
01:35:27,588 --> 01:35:31,121
need to have in your mind a
mental model of What percentage

1697
01:35:31,121 --> 01:35:34,593
of people that use your free
thing are going to upgrade to

1698
01:35:34,593 --> 01:35:38,066
your paid thing? It's not even
really to do with like open

1699
01:35:38,066 --> 01:35:41,000
source, right? But open source
makes this problem

1700
01:35:41,000 --> 01:35:45,000
Audrow Nash: Yeah. just a
conversion rate for keeping your

1701
01:35:45,000 --> 01:35:48,524
Adrian Macneil: And typically
that is going to be pretty low.

1702
01:35:48,524 --> 01:35:51,875
So you think about, a lot of,
there's a lot of open source

1703
01:35:51,875 --> 01:35:54,822
databases out there, for
example, right? So there's

1704
01:35:54,822 --> 01:35:58,577
something like InfluxDB. this is
like a time series database, and

1705
01:35:58,577 --> 01:36:02,275
then you can use like the hosted
version, or there's MongoDB, or

1706
01:36:02,275 --> 01:36:05,626
there's Elasticsearch, a lot of
these are open source, or,

1707
01:36:05,626 --> 01:36:09,208
sometimes slightly more creative
licenses more recently, Redis

1708
01:36:09,208 --> 01:36:12,790
and things, but, they have open
source database, and then like

1709
01:36:12,790 --> 01:36:16,487
they have a hosted version. But
it's a very, extremely generally

1710
01:36:16,487 --> 01:36:19,434
applicable thing, like
Elasticsearch is a full text

1711
01:36:19,434 --> 01:36:22,669
search database, or Redis, you
can use it for just about

1712
01:36:22,669 --> 01:36:26,194
anything. if a fraction of 1
percent of people that use that,

1713
01:36:26,194 --> 01:36:29,718
move on to the paid service,
that's still like a, large, it's

1714
01:36:29,718 --> 01:36:32,896
plenty, it's still like an
enormous kind of market that

1715
01:36:32,896 --> 01:36:36,593
you're talking about. robotics,
unfortunately, is not like that.

1716
01:36:36,593 --> 01:36:40,117
And, basically every, We can't
afford for only like 1 percent

1717
01:36:40,117 --> 01:36:43,815
of, of, of pro of big companies
that have raised a lot of money,

1718
01:36:43,815 --> 01:36:46,993
or, in some cases, big
multinational, car companies and

1719
01:36:46,993 --> 01:36:50,401
things like that, that, that
have enormous global presence,

1720
01:36:50,401 --> 01:36:53,926
enormous revenue. Every one of
those that sort of decided to,

1721
01:36:53,926 --> 01:36:56,930
to just use the open source
version because that was

1722
01:36:56,930 --> 01:37:00,339
sufficient for them, becomes
like lost revenue opportunity,

1723
01:37:00,339 --> 01:37:03,863
right? And that is, again, if
you're operating in an enormous

1724
01:37:03,863 --> 01:37:06,809
market with like tens of
thousands of potential, or

1725
01:37:06,809 --> 01:37:09,583
hundreds of thousands of
potential customers and

1726
01:37:09,583 --> 01:37:12,818
companies, that's fine. If
you're working in something a

1727
01:37:12,818 --> 01:37:15,880
bit more niche, like the
robotics industry, you can't

1728
01:37:15,880 --> 01:37:19,000
afford to have just like such a
tiny conversion, soon.

1729
01:37:19,000 --> 01:37:22,843
Audrow Nash: Yeah. That makes
good sense to me. Yeah. Because

1730
01:37:22,843 --> 01:37:26,559
we're a much smaller community
than like the web developers

1731
01:37:26,559 --> 01:37:27,000
say, I.

1732
01:37:27,000 --> 01:37:28,000
Adrian Macneil: or,

1733
01:37:28,000 --> 01:37:30,000
Audrow Nash: Or databases, which
yeah, are everything.

1734
01:37:30,000 --> 01:37:33,871
Adrian Macneil: is like every
company that needs a database.

1735
01:37:33,871 --> 01:37:37,484
yeah, but it's been good.
Honestly, the response to that

1736
01:37:37,484 --> 01:37:41,484
was super positive. everyone, I
would say maybe there was like

1737
01:37:41,484 --> 01:37:45,290
one or two messages that popped
up on like ROS Discourse or

1738
01:37:45,290 --> 01:37:48,903
things that were a bit upset
about it. But, yeah. On the

1739
01:37:48,903 --> 01:37:52,839
whole, pretty much everyone was
like, yeah, that makes sense.

1740
01:37:52,839 --> 01:37:54,000
It's totally fine.

1741
01:37:54,000 --> 01:37:55,000
Audrow Nash: Good.

1742
01:37:55,000 --> 01:37:57,740
Adrian Macneil: the, main
concerns that came up, maybe we

1743
01:37:57,740 --> 01:38:00,577
miscommunicated a couple of
things, but some people came up

1744
01:38:00,577 --> 01:38:03,558
and thought we were getting rid
of the desktop app, which sort

1745
01:38:03,558 --> 01:38:05,000
of me off guard cause I didn't

1746
01:38:05,000 --> 01:38:07,000
Audrow Nash: not what you're
doing at all. Yeah.

1747
01:38:07,000 --> 01:38:10,003
Adrian Macneil: fact, we're
adding more features to the

1748
01:38:10,003 --> 01:38:13,497
desktop app. but that popped up,
with a few people and we had to

1749
01:38:13,497 --> 01:38:16,881
correct that misconception. And
the other one was just, like I

1750
01:38:16,881 --> 01:38:20,048
said, is making sure there's
still a free version. So it's

1751
01:38:20,048 --> 01:38:23,433
not really any different. If you
go to our website and hit the

1752
01:38:23,433 --> 01:38:26,763
download button today, it's the
same experience that you were

1753
01:38:26,763 --> 01:38:30,093
getting three months ago. we've
still got the extension APIs.

1754
01:38:30,093 --> 01:38:33,096
That was another one. So people
were concerned that the

1755
01:38:33,096 --> 01:38:36,481
extension APIs were going away.
but, these things are, largely

1756
01:38:36,481 --> 01:38:39,811
the, the same experience. It's
just that we have a more clear

1757
01:38:39,811 --> 01:38:43,250
licensing now, where if you're
using it for up to three people,

1758
01:38:43,250 --> 01:38:46,198
in a company or in a team,
that's, free. And then over

1759
01:38:46,198 --> 01:38:48,000
that, you got to start paying
us.

1760
01:38:48,000 --> 01:38:50,000
Audrow Nash: Might make sense.
you guys need to eat

1761
01:38:50,000 --> 01:38:53,163
Adrian Macneil: we do. Yeah.
Yeah. like I said, yeah, we want

1762
01:38:53,163 --> 01:38:55,911
to create this really great
software. We want to keep

1763
01:38:55,911 --> 01:38:57,000
investing in it. And,

1764
01:38:57,000 --> 01:39:01,784
Audrow Nash: you have to exist
order do that. so I don't know.

1765
01:39:01,784 --> 01:39:06,722
Okay. That makes good sense. I,
think, like to me, even just the

1766
01:39:06,722 --> 01:39:11,505
first point of it's easier for
us to maintain. is huge because

1767
01:39:11,505 --> 01:39:16,057
guys your time, and I've been
involved with the ROS project

1768
01:39:16,057 --> 01:39:20,764
for a long time. You need the
staffing to do that, or you can

1769
01:39:20,764 --> 01:39:25,470
sour relationships too. someone
puts out a feature request or

1770
01:39:25,470 --> 01:39:30,177
something, and if you don't get
to it after a while, a little

1771
01:39:30,177 --> 01:39:35,038
bit of bad blood that's created
there this of kind if it's like

1772
01:39:35,038 --> 01:39:39,667
endless visualization features,
I can understand how kind of

1773
01:39:39,667 --> 01:39:44,065
turning that off, because the
expectation is if it's open

1774
01:39:44,065 --> 01:39:48,000
source that you're going to be
triaging everything.

1775
01:39:48,000 --> 01:39:51,084
Adrian Macneil: come up with
the, yeah, the, reports and

1776
01:39:51,084 --> 01:39:54,554
things and yeah, and we do want
to get that. And now, honestly,

1777
01:39:54,554 --> 01:39:57,749
like said, the, moving fast
thing is important too, right?

1778
01:39:57,749 --> 01:40:01,108
Like now, internally, we, just
have a single, like I said, we

1779
01:40:01,108 --> 01:40:03,917
merged a lot, all between the,
sort of data and the

1780
01:40:03,917 --> 01:40:07,277
visualization. So moving better
product experience. We're not

1781
01:40:07,277 --> 01:40:10,361
juggling like open source repo
that we're then trying to

1782
01:40:10,361 --> 01:40:13,666
incorporate into another product
and things. It's just we're

1783
01:40:13,666 --> 01:40:17,136
building this one app. people,
sign up for it, they use it. and

1784
01:40:17,136 --> 01:40:20,000
a pretty, a concept people are
pretty familiar with.

1785
01:40:20,000 --> 01:40:24,426
Audrow Nash: For sure. Yeah, I
guess like it I guess the moving

1786
01:40:24,426 --> 01:40:28,430
from open to closed it like
saddens me a little bit But I

1787
01:40:28,430 --> 01:40:32,364
understand for very pragmatic
reasons and I think it was

1788
01:40:32,364 --> 01:40:36,438
probably a very good decision
and I bet it was a very hard

1789
01:40:36,438 --> 01:40:37,000
decision

1790
01:40:37,000 --> 01:40:38,000
Adrian Macneil: I

1791
01:40:37,000 --> 01:40:40,765
Audrow Nash: making the decision
and I think it'll be better for

1792
01:40:40,765 --> 01:40:41,000
guys

1793
01:40:41,000 --> 01:40:44,804
Adrian Macneil: I was probably,
but with, every decision as a

1794
01:40:44,804 --> 01:40:48,795
company, you always think, wow,
we could have made that decision

1795
01:40:48,795 --> 01:40:52,724
a year ago or two years ago. It
was in hindsight. I think, like

1796
01:40:52,724 --> 01:40:56,154
I said, coming from an
engineering background and using

1797
01:40:56,154 --> 01:40:59,771
a lot of open source software
and I was probably the most,

1798
01:40:59,771 --> 01:41:03,762
saddened out of everyone, right?
People pop up and they're like,

1799
01:41:03,762 --> 01:41:07,379
oh, we're sad that you're
moving, and I'm like, look, I am

1800
01:41:07,379 --> 01:41:09,000
too, but but, we had to do

1801
01:41:09,000 --> 01:41:10,000
Audrow Nash: I think so.

1802
01:41:10,000 --> 01:41:12,780
Adrian Macneil: it made sense,
and I think when you talk

1803
01:41:12,780 --> 01:41:15,809
through it like that you say,
look, these are the constraints

1804
01:41:15,809 --> 01:41:17,000
that we're working with,

1805
01:41:17,000 --> 01:41:19,851
Audrow Nash: Totally. Yeah, you
guys have to stick around if you

1806
01:41:19,851 --> 01:41:22,257
devote your resources to
something that's not helping,

1807
01:41:22,257 --> 01:41:25,064
you stick around. if you, guys
went out of business, that would

1808
01:41:25,064 --> 01:41:26,000
be a much bigger loss

1809
01:41:26,000 --> 01:41:29,641
Adrian Macneil: exactly, It's
not a choice between us

1810
01:41:29,641 --> 01:41:33,695
continuing to develop and pay,
right? It's look, we want to

1811
01:41:33,695 --> 01:41:35,000
exist as a company.

1812
01:41:35,000 --> 01:41:36,000
Audrow Nash: for

1813
01:41:35,000 --> 01:41:37,000
Adrian Macneil: we're

1814
01:41:37,000 --> 01:41:39,990
Audrow Nash: it's such an
exciting time, so you definitely

1815
01:41:39,990 --> 01:41:42,000
want to exist super exciting
time. But,

1816
01:41:42,000 --> 01:41:45,000
Adrian Macneil: time to be in
robotics.

1817
01:41:45,000 --> 01:41:50,084
Audrow Nash: so we're, coming to
the end of the time. one thing I

1818
01:41:50,084 --> 01:41:54,464
wanna make sure we talk about,
tell me about the Actuate

1819
01:41:54,464 --> 01:41:59,000
Conference and, why and what
you're showing and everything

1820
01:41:59,000 --> 01:42:02,764
Adrian Macneil: yeah, so Actuate
is a one day event that we're

1821
01:42:02,764 --> 01:42:06,286
putting on in San Francisco.
It's going to be on September

1822
01:42:06,286 --> 01:42:09,807
18th. got a really nice space
that we've, venue that we've

1823
01:42:09,807 --> 01:42:13,268
booked in the dog patch in San
Francisco. and this is, an

1824
01:42:13,268 --> 01:42:17,154
event, a conference for robotics
developers. Anyone? robotics is

1825
01:42:17,154 --> 01:42:20,979
a very sort of cross functional
domain. There's a lot of people

1826
01:42:20,979 --> 01:42:24,500
working on a lot of different
things. But we want to bring

1827
01:42:24,500 --> 01:42:27,900
together engineers, across
people working on perception,

1828
01:42:27,900 --> 01:42:31,239
planning, AI, simulation,
frameworks, hardware even. We

1829
01:42:31,239 --> 01:42:34,457
want to bring together, bring
together a lot of these

1830
01:42:34,457 --> 01:42:38,039
different disciplines. It's a
one day event. We've got some

1831
01:42:38,039 --> 01:42:41,864
really great speakers lined up.
and we've got some, some really

1832
01:42:41,864 --> 01:42:45,629
great attendees lined up. The
goal of this is how can we bring

1833
01:42:45,629 --> 01:42:49,029
together just lot of the
community in the Bay Area where

1834
01:42:49,029 --> 01:42:52,793
we get to talk to. As I would
say, in the position that we are

1835
01:42:52,793 --> 01:42:56,679
in as Foxglove I get to talk to
a lot of different companies and

1836
01:42:56,679 --> 01:43:00,261
hear a lot about how they're
building their robots. none of

1837
01:43:00,261 --> 01:43:03,843
our customers talk to each
other. there's a real gap in the

1838
01:43:03,843 --> 01:43:07,668
robotics industry of, like deep
technical content on how people

1839
01:43:07,668 --> 01:43:11,432
building robots in production.
There's a lot of, there's a lot

1840
01:43:11,432 --> 01:43:14,954
of content out there from the
academic perspective, right?

1841
01:43:14,954 --> 01:43:18,718
There's a lot of papers, there's
a lot of robotics conferences

1842
01:43:18,718 --> 01:43:22,239
you can go to and listen to. a
lot of sort of the academic

1843
01:43:22,239 --> 01:43:25,882
perspective, not so much in
industry. I would say the, stuff

1844
01:43:25,882 --> 01:43:29,464
that exists in, the industry
today, in terms of, like talks

1845
01:43:29,464 --> 01:43:32,986
and conferences and things,
ROSCON is, a similar level of,

1846
01:43:32,986 --> 01:43:36,386
has the technical depth, but
it's centered around ROS, I

1847
01:43:36,386 --> 01:43:40,150
would say, ROS is, one framework
in the industry, but we see a

1848
01:43:40,150 --> 01:43:43,793
lot of, our customers not using
ROS, and we'd like to have a

1849
01:43:43,793 --> 01:43:47,436
broader perspective there. and
then you see a lot of kind of

1850
01:43:47,436 --> 01:43:50,896
robotics trade shows, like
Automate and things like that,

1851
01:43:50,896 --> 01:43:54,418
where people, you've got the
sales team, they've got their

1852
01:43:54,418 --> 01:43:58,000
robots, they're, showing off
their robots, but it's not how

1853
01:43:58,000 --> 01:44:01,582
they built the robot, right?
It's, this is just look at the

1854
01:44:01,582 --> 01:44:05,529
robot, what the robot can do. So
we felt that there was this kind

1855
01:44:05,529 --> 01:44:09,293
of, there's, a real gap for high
quality technical content, in

1856
01:44:09,293 --> 01:44:13,179
the robotics industry. Talks and
not just listening to the talks

1857
01:44:13,179 --> 01:44:16,579
but getting together and
networking with other people in

1858
01:44:16,579 --> 01:44:19,979
the industry. And I think,
because it's a one day event,

1859
01:44:19,979 --> 01:44:23,925
we're expecting it's going to be
a mostly Bay Area audience. I do

1860
01:44:23,925 --> 01:44:27,629
know some folks are flying in
for it, for the first year that

1861
01:44:27,629 --> 01:44:30,725
we're doing this, we're
expecting that we're mostly

1862
01:44:30,725 --> 01:44:34,307
going to have kind of Bay Area
attendees. I think we would,

1863
01:44:34,307 --> 01:44:37,768
look to go, a bit larger next
year, probably. And then in

1864
01:44:37,768 --> 01:44:41,593
terms of speakers, we've got a
handful of really great speakers

1865
01:44:41,593 --> 01:44:44,932
lined up. We've got, Sergey
Levine. He's a professor at

1866
01:44:44,932 --> 01:44:48,454
Berkeley and co founder of,
company Physical Intelligence.

1867
01:44:48,454 --> 01:44:52,218
We've got Chrysler Lalancette.
He's the technical lead of ROS.

1868
01:44:52,218 --> 01:44:55,314
Steve Macenski. He's the
technical lead of the NAV2

1869
01:44:55,314 --> 01:44:59,018
project. Kat Scott from Open
Robotics, I think you'll be very

1870
01:44:59,018 --> 01:45:02,782
familiar with her. Vijay from
WAVE, he's the VP of AI at WAVE.

1871
01:45:02,782 --> 01:45:06,486
So we've got a whole lot of good
speakers already, and then I

1872
01:45:06,486 --> 01:45:10,129
think we have a number of, we
currently actually have a call

1873
01:45:10,129 --> 01:45:13,589
for proposals open right now.
Number of super interesting

1874
01:45:13,589 --> 01:45:17,171
proposals we've had come in.
Like I said, we want to really

1875
01:45:17,171 --> 01:45:20,814
find a great balance of content
across people working on AI,

1876
01:45:20,814 --> 01:45:24,518
people working on, perception
and planning, people working on

1877
01:45:24,518 --> 01:45:28,282
simulation, just cover the whole
sort of spectrum of different

1878
01:45:28,282 --> 01:45:31,986
things that, that we do. that
are needed in robotics and help

1879
01:45:31,986 --> 01:45:35,629
encourage them sharing, have
some really great speakers, but

1880
01:45:35,629 --> 01:45:39,150
also have really great attendees
and, high quality kind of

1881
01:45:39,150 --> 01:45:40,000
conversations.

1882
01:45:40,000 --> 01:45:44,603
Audrow Nash: Do you, is there
much planned for the evening

1883
01:45:44,603 --> 01:45:45,000
after

1884
01:45:45,000 --> 01:45:46,000
Adrian Macneil: Yeah.

1885
01:45:45,000 --> 01:45:49,214
Audrow Nash: I find like the
unstructured time to just talk

1886
01:45:49,214 --> 01:45:53,000
with all the me, that's my
favorite part of the whole

1887
01:45:53,000 --> 01:45:56,156
Adrian Macneil: Yeah. And I
also, I, yeah, that, yeah,

1888
01:45:56,156 --> 01:45:59,838
there's, structured lunchtime.
There's a cocktail hour. I think

1889
01:45:59,838 --> 01:46:03,637
it's from four to six or four to
seven or something like that. So

1890
01:46:03,637 --> 01:46:07,319
we've got like a happy hour at
the end as well. so there, there

1891
01:46:07,319 --> 01:46:10,651
will be plenty of kind of
unstructured time for chatting.

1892
01:46:10,651 --> 01:46:14,333
Also, just a single track event
is a really fun thing. I think,

1893
01:46:14,333 --> 01:46:17,197
a lot of these larger
conferences, get to a point

1894
01:46:17,197 --> 01:46:20,704
where there's all these multiple
talks and there's rooms and

1895
01:46:20,704 --> 01:46:24,035
presentation, there's a hall
with all of the, vendors and

1896
01:46:24,035 --> 01:46:27,834
things and you spend most of the
conference wandering around. but

1897
01:46:27,834 --> 01:46:31,633
I think the, beauty of a, single
track event is that, there'll be

1898
01:46:31,633 --> 01:46:35,315
nice punchy talks. You come on,
hey, we're having this session,

1899
01:46:35,315 --> 01:46:38,764
just like everyone listen, and
then everyone can go out and

1900
01:46:38,764 --> 01:46:42,329
we'll have some, some time to
chat and get to know people. so

1901
01:46:42,329 --> 01:46:45,661
yeah, we absolutely like the
networking times good, but I

1902
01:46:45,661 --> 01:46:49,109
also think the beauty of a
single track event is that these

1903
01:46:49,109 --> 01:46:52,499
have any of questions of oh,
which talk should I go to? Or

1904
01:46:52,499 --> 01:46:56,181
should I just maybe I should be
networking instead of listening

1905
01:46:56,181 --> 01:46:59,688
to the talk and yeah, it gets a
bit, many Too Yeah, it'll be

1906
01:46:59,688 --> 01:47:03,019
fun. It'll be awesome. yeah,
we'd love to have you there,

1907
01:47:03,019 --> 01:47:06,000
Audrow, and, yeah, we will,
we'll make that happen.

1908
01:47:06,000 --> 01:47:10,500
Audrow Nash: Hell yeah. Yeah, I
would love to go. if it works

1909
01:47:10,500 --> 01:47:15,000
out, I think I'll be there. I
think you had some code with my

1910
01:47:15,000 --> 01:47:18,335
Adrian Macneil: Absolutely.
Yeah, So for folks listening

1911
01:47:18,335 --> 01:47:21,729
that have made it this far in
the podcast, so a couple of

1912
01:47:21,729 --> 01:47:25,362
things. One is we currently have
early bird, like I said, the

1913
01:47:25,362 --> 01:47:28,637
event's on September 18th. It's
a single day event, San

1914
01:47:28,637 --> 01:47:32,389
Francisco. it's from about 9 a.
m. or 9. 30 to, 4, or something

1915
01:47:32,389 --> 01:47:36,081
like that. We have early bird
ticket sales going on right now.

1916
01:47:36,081 --> 01:47:39,833
So I think the first hundred and
fifty tickets that we sell are

1917
01:47:39,833 --> 01:47:43,167
150 dollars or 149, something
like that. So get in quick

1918
01:47:43,167 --> 01:47:46,860
before those sell out. And then
we have an additional, I think

1919
01:47:46,860 --> 01:47:50,075
on top of that, it'd be
additional 20 percent discount

1920
01:47:50,075 --> 01:47:53,648
for listeners of the podcast.
Just put code Audrow in there.

1921
01:47:53,648 --> 01:47:57,281
Check out A U D R O W. that's
the secret code, which will get

1922
01:47:57,281 --> 01:48:00,735
you an additional 20 percent
off, 20 percent off the early

1923
01:48:00,735 --> 01:48:04,248
bid price if you make it in
time, or it will be, 20 percent

1924
01:48:04,248 --> 01:48:07,643
off the full price if you're a
little bit slow, so get in

1925
01:48:07,643 --> 01:48:08,000
quick.

1926
01:48:08,000 --> 01:48:13,056
Audrow Nash: Hell yeah. Awesome.
so with that, I think we shall

1927
01:48:13,056 --> 01:48:18,272
wrap up. Great talking with you,
Adrian. So cool Foxglove and all

1928
01:48:18,272 --> 01:48:21,000
the great work you guys are
doing.

1929
01:48:21,000 --> 01:48:22,000
Adrian Macneil: Absolutely,
yeah,

1930
01:48:22,000 --> 01:48:23,000
Audrow Nash: see you at

1931
01:48:22,000 --> 01:48:25,000
Adrian Macneil: As always,
thanks, Audrow.

1932
01:48:25,000 --> 01:48:28,121
Audrow Nash: All right. Bye
everyone. That's it! You made

1933
01:48:28,121 --> 01:48:31,296
it! What has your experience
been with Foxglove? Or if you

1934
01:48:31,296 --> 01:48:34,472
haven't used it, do you think
you'll give it a try? If you

1935
01:48:34,472 --> 01:48:37,483
like this interview, I'd
appreciate a like or a review.

1936
01:48:37,483 --> 01:48:40,440
It helps other people find the
podcast and pleases the

1937
01:48:40,440 --> 01:48:43,779
algorithms. And if you'd like to
make sure you hear about new

1938
01:48:43,779 --> 01:48:47,338
episodes, make sure to follow or
subscribe. Our next episode is a

1939
01:48:47,338 --> 01:48:50,678
very strong robotics company in
the food space. They're doing

1940
01:48:50,678 --> 01:48:53,853
great work with robotics and
machine learning, and they've

1941
01:48:53,853 --> 01:48:57,303
already done over 5 million food
servings in production. That's

1942
01:48:57,303 --> 01:48:59,000
all for now. See you next time.
