I would say: This is a precursor to robot skins And that you like to use "weird" a lot, like you're trying to induce people to subtrans state or something, plus flashing lights, red pipe, flags all around
I see special applications. Couple it with a surface duplication device, and it could be used to allow a blind person to "see" the tiny details they would not normally be able to detect by touch. For surface duplication; en.m.wikipedia.org/wiki/Pin_Art A modified "Pin Art" board, with micro solenoid control for each individual pin, and a similar membrane over the top. The "imager" could then be pressed onto a tiny object, and a much larger magnified "image" replicated using the PinArt-based tactile "display".
I'd love to know more about the gel they are using, It seems absolutely incredible that a gel would conform to such small details so perfectly and yet have no memory in that it returns to it's original shape.
To be clear, they do have to replace these pads with varying frequency depending on what the application is. They wear and could potentially leave behind debris on the part being measured.
Really? It's kinda natural and comment sense that a semi solid/liquid would form to anything that is placed on it with pressure or it pressed into something with pressure... What's so "unforeseen" to you?
I could probably watch about 12 hours of Steve pressing various objects into the gel and just describing what they are before moving onto the next one.
It's actually nuts that you're getting such a deep depth of field on such small things, and you can even move them around a few millimeters and still retain focus. And you can capture it all with video and then share it with other people. Amazing.
That's a really good point. My experience with regular microscopes is that the focus is touchy and the depth of field is very shallow, unlike here where the whole object is in focus.
The wierd neon-colored image you can see used to create 3D model is called a normal map and it's meant to hold information about height of the object using different colours for different light angles. It is used widely in video games to create detail in low poly objects by changing how light reflects off of the surface. I find it fascinating they used normal map and not a regular height map but given the technology of 6 light sources it makes all the sense
Height maps contain less information than normal maps. Each pixel's color channel in a normal map contains information on the normal (orientation) of the surface, whereas a height map only contains direct vertical information.
This is insane. We are getting to watch the first steps of an entire new tool that will change entire fields of study. It is amazing. I'm probably way more hyped about this then I should
We could actually use that almost every day at work to measure engravings on the toolings for chocolate moulds. Would safe us a lot of time. Super interesting 🧐👌🏼
As a 3D artist myself this was one of the most fascinating things I have ever seen in a long while. If produced massively across the globe. It could be really helpful in 3D Film Industry.
You can bet your ass the artists who worked on the Lego movie would have killed to get one of these. I wonder if they could make a giant one and go press it on peoples face to extract a bump map :D
Was really wondering about how easily it was contaminated and how frequently you would need to replace the gel, how expensive is the replacement and how easy is the process? It looked like everything you had on there had some dust or hairs, it seemed like it wasn't like you were constantly scanning dirty pieces but instead that the microscope itself had issue. Incredibly cool technology, I even want one but the dirt issue seems hard to deal with.
This has WILD implications as a mobile highly accurate surface measurer. Damn, I want one just to have. I bet they're expensive. Industry really needs to have this. Feels revolutionary.
Ok, Steve, I'm so glad that youtube thought to put your video in my path. That's an odd microscope indeed, and the gel is strangely sensitive. Removing color does indeed leave us with form, texture, and shadow. Photometric Stereo? Ok then. I wonder what insects would look like with this system? Does that gel wear out? So it has a quality control inspection use, also interesting. Jane Street Academy... sounds excellent. Thank you for all of this information. Well done. But then you already know that.
This would be awesome to generate height maps for making smart materials in substance. Just noticed the height map at 2:41 and wanted to plug it into substance. Mistook height map and normal map as the same thing by mistake here. Thankfully people seem to get what I meant
@Esger My engineer thesis was really similar. But I used 0,5x0,5x0,5m frame, 4 sources of lights and camera. Results were miserable but really good for a price I made that device (I think it was about 30$ not counting my own 150$ camera) I worked with textures but never came up with to merge that two ideas :D
I work with 3D making games for a living. Seeing those 3D models being created was so cool. Essentially using the same information that's used to generate and render normal and displacement maps.
what a *fascinating* tool. I personally don't have a lot of use for it, but I can just imagine machinists being absolutely delighted at the details and measurements you can get from it.
That mystery item looks like a watch battery and the surrounding parts, with a lot of overlapping layers of parts held in by small screws, I'm going to guess it's a complex mechanism... so, a watch? The movement of an analog quartz wristwatch? Edit: I realized after posting that I betrayed my knowledge of watchmaking with the terminology. For those who don't know, the "movement" of a watch refers to the mechanism, specifically the moving parts that make it work, the gears, the springs, and so on. In a quartz watch, it'd also include the battery and other electronic parts. The hands are also included but obviously not shown here.
You can take a bunch (like 50 to 100) photos of an object and load them in to 'Meshroom'. It's a photogrammetry program that gives you a 3D model that includes colour. You can then load that model in to blender or some other program. It does have a hard time with verry reflective and translucent objects though.
@Asu Kan it would complicate the process tremendously if there were more lights. If what you are scanning is relatively small then an older chunkier flatbed scanner can be used as they have some depth focus and a light that's offset to the side. Just scan 4 times, rotating 90 degrees each time
@Martin Baadsgaard Does it have to be in a studio setting i.e., *only* light from the sides and dark from everywhere else or just increased light from the sides?
Never knew such thing existed. But this is really a valuable tool for a lot of people who do precision work. So thank you for brining it to my attention it deserves that.
This multi-light setup that generates the depth and normal maps is the same method that is used to generate many textures and material used in games and film. You can do it yourself with a camera, a light and either free software or Adobe Substance. You can also remove the reflection from the images using cross polarisation by putting a polarising gel on your light source and in a different orientation on your camera.
Sounds like it could be a major improvement to a thumbprint scanner. I’m a mechanic and get cuts and callouses on my hands every day. My thumbprint usually only lasts 2 days on my iPhone before I have to rescan it.
haha I'm surprised to see someone other than myself comment on this within the top comments =D I hope he does check out or try to make his own atomic force microscope hehe (well, something analogous to one)
I was also expecting an AFM, but I found this less fascinating. The magnification is just very minimal, you can see all the details here with the naked eye.
This reminds me of when my dad, a master carpenter, taught me to feel if 2 pieces are level on a small scale with my fingers instead of looking because it's way more accurate.
I've always had an issue viewing the crater of Neal Armstrong's bootprint on the moon. I always see the boot print as raised above the surface. From the knowledge gained in this video, I flipped the image 180 degrees, and voila, now I see the imprint instead. Amazing!
I have 16 years as a US military aircraft structural mechanic. I love your presentation style. It's light and engaging. I have never seen our nondestructive technicians make a cast of an indentation. We use ultrasound, edy current, sometimes x-rays, and more rarely these days optical micrometers. Never the less. I need one of these, I would love to compare it to currently accepted techniques.
That zigzag motion of the print head is "boustrophedonic". It is a literal reference to the motion of an ox ploughing a field, and you still usually see it in patents to describe print heads, scanners, etc.
The same technology is used in AOI machines worldwide to provide 3d models of the inspected board. Such precision is used to find out if components were assembled wrong, have defects got dislodged while going through the oven. Edit: those don't use gel though.
We have a 3d scanner at work that works on a similar, off less accurate, principle of shining a light onto something and taking a picture. It is amazing how it will confuse printed text as raised or lowered surface. This seems to be a great workaround the normal "spray paint it all grey" method we have to use. about a decade ago i worked with a charity to try and replicate a giant version of a medal. I tried photogrammetry to get a scan of a plaster cast of the medal (again, all uniform no shiny colour). it ended up terrible. Something like this would have worked amazing.
Nailed it! I recognized the battery from the mystery device after a bit of pondering, and then just guessed the most common application of that kind of battery.
This would be an incredible medium to make some kind of animated movie, even just a short little thing. What i'm imagining is in the same vein as the "A Boy And His Atom" animation made with individual atoms.
A note on the ML side - the thing Steve refers to is called "top down" reasoning, where the later "high-level" layers representing object types feed back into the earlier "low-level" layers representing shapes and edges. This is fantastically computationally expensive, and is one indicator of just how sophisticated the human visual system is.
There seems to be some limitations when used for surface features with high aspect ratio. For example, it wasn't able to discern the vertical fall off from the edges of the Lego brick pegs But none the less very interesting and could be very useful in a lot of scenarios By the way, what's the MTBF of the gel? (i.e. how many times can it be used)
The crater illusion is tripping me out because of how inconsistent I'm finding it between people. The crater image only ever looked like a crater to me, while many commenters had the opposite experience. Then with the indented lettering, it looked indented to me the first way, and like it was sticking out while Steve is saying it obviously looks indented now XD That's the weird part to me.
I saw opposite mountains when he said crater and crater when he said mountain then immediately after the indented print looked popped out and visa versa
If possible, it would be interesting to see an example of how the algorithm would interpret a color image. It would demonstrate the gap between how these algorithms interpret images compared to our brains.
If you're interested in looking at other unique sensors, Event-Based Cameras (also called neuromorphic cameras or silicon retinas) are an interesting rabbit hole to go down. They use a unique approach to electro-optical sensing to get high fidelity of certain information in a very efficient manner.
With the crater illusion, I saw it as a mountain the whole time, even after seeing the shadows falling on it like a crater - the look of the inside of the crater looks so plateu like. Same with the indented text, in fact I think the top light source makes it stronger, and I did eventually recognise the crater, but I had to look closely at the shadows, and looking at the light side makes it look like a mountain no matter what.
He's attributing way too much importance to the "lit from above" assumption. We are all very used to seeing images shot from above where that doesn't apply. I saw the crater as a plateau in both orientations, and saw the print as indented / raised opposite to what he thought we should.
So, these replies: do the commenters regularly view satellite imagery? Are those images more typically from the northern or southern hemisphere? Do the commenters work in specialist lighting conditions? I'm sure someone could wring a doctorate out of studying the various ways humans interpret relief patterns and sociological effectors on that.
Yeah it was a mountain for me both directions. Then after reading this comment I went back up and rewound the video to when it was supposed to be in crater mode, and... it was a crater. I let it flip back to mountain mode, and it was still a crater. So for me at least, it seems to get "locked" as how I last saw it, regardless of rotation.
I work with lots of 3D LIDAR scans and one huge issue for us is the reflection off of reflective surfaces such as mirrors. Interseting how this gets rid of that, at least on a micro scale.
Would training an algorithm be easier using this technique as a source of truth? Small images, and maybe large images later, could be developed accurately with this, but would that information from this technique help you train a model faster? And further, would information about small things like the quarter or matchhead be useful for larger scale applications of LIDAR such as autonomous vehicles?
I think my reaction to your example at 6:00 brings up an interesting point. See, I DIDNT experience the crater illusion when you used the touch microscope, and I think it's because humans are VERY good at context clues and learning; Because you'd shown me other visuals of the touch microscope, I was able to tell from the patterning around the letters that a flat, intended surface was pressed against the microscope, and that the letters were smooth because nothing was touching the gel surface. If they really had been protruding from the block, it would be the letters that had texturing, not the background.
IAs VFX artist I love that it creates normal textures. That´s a totally cool device for scanning hightmaps and normal maps. Especially for fabrics and general surface imperfections. Exciting technology
Nearly every mundane object under that type of imaging looks quite mesmerizing! I actually thought your stubble was one of the more interesting ones. It really shows just how cleanly the blades cut the hairs at the ends. The draping problem was the first thing that came to mind as someone who has done a lot of vacuum forming. But that’s very interesting that it’s not much of an issue as long as they can get the depth value. And as long as it’s not deeper than it is wide.
The "crater" looked like a mountain to me no matter the angle. Even after you said it was a crater and flipped it, my brain couldn't un-mountain it. :D
6:07 "This image looks like it's sticking out because the shadows are on the bottom, but by flipping the light source you see the reality that they're actually indented" That was the exact opposite for me. The first instance was clearly indented while the second was and emboss.
Very interesting. I can visualize this technology being useful in coin collecting. Replacing an individual's perception on flaws or abnormalities that make a coin valuable, a definitive definition would bring consistency.
Wow this is an interesting use-case, but who would use it? regular collectors or like curating organizations? I don't know the field, so I'm not entirely sure if there's like a central authority organization or the like
as a watch guy int interesting to see a miyota(owned by citizen watch company) quartz movement inside of a casio watch who can and does make their own quartz modules
4:37 That's called monocular depth estimation and it very much already exists. There are quite a few open-source models that can estimate depth fairly well from a single flat image, and if you have multiple images from different angles there are neural networks that can build up an accurate model of the entire scene, sufficient to move a virtual "camera" around and through the scene and create new views in the process.
this is really interesting! I just learned about vanta-black objects. humans cant exactly figure out what shape vanta black is just by looking at it so it would be nice seeing vanta-black objects being used
Photometric Stereo is the proper name for the algorithm used like Steve said (comes from the original 1980 paper), but the name can be misleading, because it does not use stereo camera system (e.i. a camera pair), only a single camera. It is based upon the better named Shape From Shading algorithm because it reconstructs the depth information from shading, as opposed to camera motion, positional disparity between cameras, etc. Saying that this microscope uses touch is a bit of a stetch (I think it's arguable either way), this device is mostly optical, but interestingly there ARE microscopes which directly measure by touching the object called scanning probe microscopes. There's a neat video about an atomic force microscope on the Breaking Taps channel.
I've been following this company for many years. Their technology is highly interesting. I'm mostly interested for bio-inspired designs and collecting surface profilometry of scales, scutes, skin, etc. A disadvantage of the method is that the surface of the gel degrades with use. Perhaps they are trying to improve this design, but I think it is probably unavoidable to some extent. That consumable cost is kind of expensive for casual use, thus restricting this method to only industrial uses.
This is honestly amazing! You're so lucky to be able to play with. Is there any information on whether this is going to be sold commercially? I'd love to get my hands on this.
@Barrie Shepherd i mean , that happens with pretty much everything, the Spot robot of Boston Dynamics is 75k meanwhile the chinese knockoffs are way less expensive, even if both were of the same Quality , the original Spot would cost more due to all of the R&D involved , in the other hand the knockoff doesn't has that much R&D into it , and is less expensive due to that
I once received a quote for one about 4 years ago. It is in the tens of thousands to own. They offered a rental arrangement that is in the thousands. I don't know how much the gel pads cost, but they are consumable.
06:55 On an aircraft "Traditionally you would create a cast of the scratch and then measure the cast in the lab" - I never saw that. We could use ultrasonic inspection or other forms of non-destructive testing in-situ to measure the depth of the scratch. In any case, best to smooth it out to avoid stress concentration and the risk of crack initiation - re-checking to ensure that the panel thickness is within tolerance or safe limits.
Interesting, the images that the 3d scanner (I think that's maybe more apt than a microscope) puts out already reminded me a lot of normal maps, tech we use in games to cheaply render 3d geometry on a flat polygonal surface. Which actually look exactly like the kind of purple image you see at 2:42 . Funny to see this tech show up in unexpected places
This should be used for handprint-scanning door security access. It'd be hard to fake it. You'd need the actual hand, not just an image of the hand. I love how sci-fi shapes real technology.
Steve. Look up NeRFs! You can do 3d photogrammetry and capture reflective objects. It's a bit different than you described doing but still relevant I think. Unless you mentioned them and I missed it.
when you were explaining the optical illusions of indented and sticking out parts, my brain works exactly opposite for some reason. the ones you called indented looked sticking out to me, and the ones you called sticking out looked indented. interesting...
@Flavio D. UtzeriSame, I'm doing my bachelor's thesis on teeth abrasion due to nonmasticatory (not chewing) behavoiur, I think the results of such a scan could be useful
@Mikkel Højbak I was mainly thinking about markings on rustued metal, but on the other hand if you take (for example) a coin, you just scan both sides and you don't have to handle the coin nearly as much, which would make it far less likely to crumble.
@Mikkel Højbak well I can imagine a lot of situation in which force it's not a problem. There are some artifacts, like terra sigillata (I don't remember now the english name, sorry) which may not suffer. Or stoneware
Apart from the extremely interesting features and applications of this idea, one of my takeaway messages is the nth confirm that Hexagons are the Bestagons 🤓
I’m wondering how big this ‘microscope’ can get. If the depth can’t be more than the width, it would mean bigger microscopes would have more overall use if it were wider/bigger i’d assume
So far my biggest problem with AI, is it seems to be the new fantasy we cling to about salvation. Also for all the problems ai creates the offered solution is often just more ai lol
hi 3d artist/developer: these are SSBump maps though, more like 2x SSBumps, as SSBs use 3 light sources (packed into the R G and B channels of an image) rather than 6 3 is good enough for realtime rendering its rather hilarious to see "machine learning" mentioned in this regard because SSBump maps are below dead stupid simple. I do think this device is a *far* better visualization and explanation of how & why SSBump maps work (than the whitepapers ive seen on them anyway) and if i ever need to explain them i might use this video
I saw this (specifically the tiny GelSight mini) recently and thought "what would anyone use this for?". Now that I actually see the accuracy and resolution it has it makes sense
Well AIs can already create depth maps just from images. But of course not with the fidelity and resolution of this device. On the other hand you cannot use a gigantic gel pad to create a depth map of a room. :D
Would *you* call it a microscope?
The sponsor is Jane Street. Find out about their Academy of Math and Programing here: janestreet.com/amp
I would say:
This is a precursor to robot skins
And that you like to use "weird" a lot, like you're trying to induce people to subtrans state or something, plus flashing lights, red pipe, flags all around
Tactile microscope?
touchascope
I see special applications. Couple it with a surface duplication device, and it could be used to allow a blind person to "see" the tiny details they would not normally be able to detect by touch. For surface duplication; en.m.wikipedia.org/wiki/Pin_Art
A modified "Pin Art" board, with micro solenoid control for each individual pin, and a similar membrane over the top. The "imager" could then be pressed onto a tiny object, and a much larger magnified "image" replicated using the PinArt-based tactile "display".
I would call it the world's most accurate reverse engineering device.
I'd love to know more about the gel they are using, It seems absolutely incredible that a gel would conform to such small details so perfectly and yet have no memory in that it returns to it's original shape.
I love how someone who actually works for the manufacturer is completely ignored, while he’s stating what your asking
To be clear, they do have to replace these pads with varying frequency depending on what the application is. They wear and could potentially leave behind debris on the part being measured.
And what the gel tastes like
Probably aerogel
Really?
It's kinda natural and comment sense that a semi solid/liquid would form to anything that is placed on it with pressure or it pressed into something with pressure...
What's so "unforeseen" to you?
I could probably watch about 12 hours of Steve pressing various objects into the gel and just describing what they are before moving onto the next one.
Right? I wonder what various kinds of feathers would look like - flight feathers vs down, etc
It's actually nuts that you're getting such a deep depth of field on such small things, and you can even move them around a few millimeters and still retain focus. And you can capture it all with video and then share it with other people. Amazing.
That's a really good point. My experience with regular microscopes is that the focus is touchy and the depth of field is very shallow, unlike here where the whole object is in focus.
The wierd neon-colored image you can see used to create 3D model is called a normal map and it's meant to hold information about height of the object using different colours for different light angles. It is used widely in video games to create detail in low poly objects by changing how light reflects off of the surface.
I find it fascinating they used normal map and not a regular height map but given the technology of 6 light sources it makes all the sense
Height maps contain less information than normal maps. Each pixel's color channel in a normal map contains information on the normal (orientation) of the surface, whereas a height map only contains direct vertical information.
This is insane. We are getting to watch the first steps of an entire new tool that will change entire fields of study. It is amazing. I'm probably way more hyped about this then I should
Not at all! We're curious beings and seeing new innovations inspires our mindset!
We could actually use that almost every day at work to measure engravings on the toolings for chocolate moulds. Would safe us a lot of time.
Super interesting 🧐👌🏼
As a 3D artist myself this was one of the most fascinating things I have ever seen in a long while. If produced massively across the globe. It could be really helpful in 3D Film Industry.
@Alan Berger you can only sell something for what the buyer is willing to pay in a free society, have you forgotten?
@torhgrim 😂😂😂 LOL Yup. I want this so desperately. making 3d visuals for my channel.lol would love to see your feedback.
You can bet your ass the artists who worked on the Lego movie would have killed to get one of these. I wonder if they could make a giant one and go press it on peoples face to extract a bump map :D
@Joseph Still $500 is for the hobby "mini" version. Steve was using the professional version.
Was really wondering about how easily it was contaminated and how frequently you would need to replace the gel, how expensive is the replacement and how easy is the process? It looked like everything you had on there had some dust or hairs, it seemed like it wasn't like you were constantly scanning dirty pieces but instead that the microscope itself had issue. Incredibly cool technology, I even want one but the dirt issue seems hard to deal with.
This has WILD implications as a mobile highly accurate surface measurer. Damn, I want one just to have. I bet they're expensive. Industry really needs to have this. Feels revolutionary.
I would love a larger scale of this, it would be amazing for modelling small electronics where the manufacturer doesn't provide 3d files
So cool. Its like making a normal map of a object 😍
Ok, Steve, I'm so glad that youtube thought to put your video in my path. That's an odd microscope indeed, and the gel is strangely sensitive. Removing color does indeed leave us with form, texture, and shadow. Photometric Stereo? Ok then. I wonder what insects would look like with this system? Does that gel wear out? So it has a quality control inspection use, also interesting. Jane Street Academy... sounds excellent. Thank you for all of this information. Well done. But then you already know that.
This would be awesome to generate height maps for making smart materials in substance.
Just noticed the height map at 2:41 and wanted to plug it into substance.
Mistook height map and normal map as the same thing by mistake here. Thankfully people seem to get what I meant
& 6:30 profit 2 it’s a pity that the aftor didn’t put the tip of a ballpoint pen there or traces of the electronic board or damage on the wires
@Paco That might be tricky to pull off. The slightest movement between shots will ruin the necessary alignment.
@Esger My engineer thesis was really similar. But I used 0,5x0,5x0,5m frame, 4 sources of lights and camera. Results were miserable but really good for a price I made that device (I think it was about 30$ not counting my own 150$ camera) I worked with textures but never came up with to merge that two ideas :D
I was just saying "it looks like a height map" and then I scroll down and see this. Artists thinks alike
That's what they've been doing AFAIK
I work with 3D making games for a living. Seeing those 3D models being created was so cool. Essentially using the same information that's used to generate and render normal and displacement maps.
I work at a jet engine company and we use these all the time! So cool to see!
to what end? wear inspection?
what a *fascinating* tool. I personally don't have a lot of use for it, but I can just imagine machinists being absolutely delighted at the details and measurements you can get from it.
That mystery item looks like a watch battery and the surrounding parts, with a lot of overlapping layers of parts held in by small screws, I'm going to guess it's a complex mechanism... so, a watch? The movement of an analog quartz wristwatch?
Edit: I realized after posting that I betrayed my knowledge of watchmaking with the terminology. For those who don't know, the "movement" of a watch refers to the mechanism, specifically the moving parts that make it work, the gears, the springs, and so on. In a quartz watch, it'd also include the battery and other electronic parts. The hands are also included but obviously not shown here.
I imagine this could be incredibly useful for making normal maps for added detail in 3d modeling
im a 3d artist and i want one of these to make instant normalmaps from objects ,these look great
You can take a bunch (like 50 to 100) photos of an object and load them in to 'Meshroom'. It's a photogrammetry program that gives you a 3D model that includes colour. You can then load that model in to blender or some other program. It does have a hard time with verry reflective and translucent objects though.
@Multiarray Haha did it?! xD
@Asu Kan it would complicate the process tremendously if there were more lights. If what you are scanning is relatively small then an older chunkier flatbed scanner can be used as they have some depth focus and a light that's offset to the side. Just scan 4 times, rotating 90 degrees each time
@Martin Baadsgaard Does it have to be in a studio setting i.e., *only* light from the sides and dark from everywhere else or just increased light from the sides?
I would use it for displacement rather. Normal maps are useful only for realtime rendering.
Never knew such thing existed.
But this is really a valuable tool for a lot of people who do precision work.
So thank you for brining it to my attention it deserves that.
I could imagine a larger version of this combined with a traditional 3d scanner to very quickly generate texture maps for game design
Steve: this needs a sound effect
Me: schlorp
Steve: schlorp
This multi-light setup that generates the depth and normal maps is the same method that is used to generate many textures and material used in games and film. You can do it yourself with a camera, a light and either free software or Adobe Substance. You can also remove the reflection from the images using cross polarisation by putting a polarising gel on your light source and in a different orientation on your camera.
Sounds like it could be a major improvement to a thumbprint scanner. I’m a mechanic and get cuts and callouses on my hands every day. My thumbprint usually only lasts 2 days on my iPhone before I have to rescan it.
I was expecting a video on an Atomic Force Microscope, but this is far more fascinating! What a novel way to collect 3d data!
Heck out breaking taps channel
As a microscope enthusiast and former microscope student i agree this is really cool
haha I'm surprised to see someone other than myself comment on this within the top comments =D I hope he does check out or try to make his own atomic force microscope hehe (well, something analogous to one)
same
I was also expecting an AFM, but I found this less fascinating. The magnification is just very minimal, you can see all the details here with the naked eye.
This reminds me of when my dad, a master carpenter, taught me to feel if 2 pieces are level on a small scale with my fingers instead of looking because it's way more accurate.
I've always had an issue viewing the crater of Neal Armstrong's bootprint on the moon. I always see the boot print as raised above the surface. From the knowledge gained in this video, I flipped the image 180 degrees, and voila, now I see the imprint instead. Amazing!
I'm now wondering: Isn't this microscope still optical since it uses light to create the shadows that form the picture?
I have 16 years as a US military aircraft structural mechanic.
I love your presentation style. It's light and engaging.
I have never seen our nondestructive technicians make a cast of an indentation. We use ultrasound, edy current, sometimes x-rays, and more rarely these days optical micrometers.
Never the less. I need one of these, I would love to compare it to currently accepted techniques.
I’m surprised they don’t use vacuums or air pressure to get the gel to conform more to the objects
That zigzag motion of the print head is "boustrophedonic". It is a literal reference to the motion of an ox ploughing a field, and you still usually see it in patents to describe print heads, scanners, etc.
I learned what that word meant in the context of writing systems.
Space filling curves
I'm here to learn where you learned this
This concludes my night, I've learnt enough. Thank you for that nugget of information.
I'm here to learn words like boustrophedonic
The same technology is used in AOI machines worldwide to provide 3d models of the inspected board.
Such precision is used to find out if components were assembled wrong, have defects got dislodged while going through the oven.
Edit: those don't use gel though.
We have a 3d scanner at work that works on a similar, off less accurate, principle of shining a light onto something and taking a picture. It is amazing how it will confuse printed text as raised or lowered surface. This seems to be a great workaround the normal "spray paint it all grey" method we have to use.
about a decade ago i worked with a charity to try and replicate a giant version of a medal. I tried photogrammetry to get a scan of a plaster cast of the medal (again, all uniform no shiny colour). it ended up terrible. Something like this would have worked amazing.
This is like real life normal mapping. Super cool
My past 5+ years of EM experience using various electron detectors approves this video on a fundamental level.
Its incredible how well that gel conforms to objects
@MEMES FROM DEEP SPACE He actually did say that in the video, on the part with the stickiness. He said that it becomes sticky and needs replacing :)
I bet the gell have "working life time" ..... The gell Will be the thing Will replace regulary ..... 🤔🤔🤔
I think it is the 80% of that technology.
I don't know about a microscope, but we'd surely hope to see this tech being used in creating 3D models
Nailed it! I recognized the battery from the mystery device after a bit of pondering, and then just guessed the most common application of that kind of battery.
I replaced a watch battery a month ago and remembered that. Lol
Thanks for using an actual stereographic video and making it easy to cross my eyes to see that sea cucumber swim in 3D.
Got to play with one last year. They're 40k. We were using it to look for pinholes in our product, but I mostly used it for coins
This company presented us this technology at work, it was satisfying and playful to try 😀
This would be an incredible medium to make some kind of animated movie, even just a short little thing. What i'm imagining is in the same vein as the "A Boy And His Atom" animation made with individual atoms.
@JustOneAsbesto You aren't very bright huh?
@JustOneAsbesto I mean isn't technically everything that it sees made of atoms?
@Nick Coleman yo thats perfect!
‘A Boy and His Micron’ maybe?
@JustOneAsbesto I was not suggesting that it did, thanks though.
A note on the ML side - the thing Steve refers to is called "top down" reasoning, where the later "high-level" layers representing object types feed back into the earlier "low-level" layers representing shapes and edges. This is fantastically computationally expensive, and is one indicator of just how sophisticated the human visual system is.
Does the gel leave a residue behind to the point to where you have to refill the gel over time?
There seems to be some limitations when used for surface features with high aspect ratio. For example, it wasn't able to discern the vertical fall off from the edges of the Lego brick pegs
But none the less very interesting and could be very useful in a lot of scenarios
By the way, what's the MTBF of the gel? (i.e. how many times can it be used)
I'd probably call it a profilometer more than a microscope. But then again it can be one
Oh... you mentioned it towards the end of the video... hahaha
It's surprising that the gel doesn't capture more bubbles of air
The crater illusion is tripping me out because of how inconsistent I'm finding it between people.
The crater image only ever looked like a crater to me, while many commenters had the opposite experience. Then with the indented lettering, it looked indented to me the first way, and like it was sticking out while Steve is saying it obviously looks indented now XD
That's the weird part to me.
I still saw a mountain even with the image turned upside down. Didn't see a crater-
Me 2
Just imagine a light source from the upper left corner, that way the shadows make it look like mountain/crater.
I can't even tell what it is 💀
I also saw the lettering as indented in both images
I saw opposite mountains when he said crater and crater when he said mountain then immediately after the indented print looked popped out and visa versa
If possible, it would be interesting to see an example of how the algorithm would interpret a color image. It would demonstrate the gap between how these algorithms interpret images compared to our brains.
The crator illusion worked opposite for me. Looked indented when lit from below and sticking out from the top
If you're interested in looking at other unique sensors, Event-Based Cameras (also called neuromorphic cameras or silicon retinas) are an interesting rabbit hole to go down. They use a unique approach to electro-optical sensing to get high fidelity of certain information in a very efficient manner.
Makes it look like some sort of height map you can generate from 3D objects
This technology can be insanely useful for 3D artists, although not this one since it's a microscope
With the crater illusion, I saw it as a mountain the whole time, even after seeing the shadows falling on it like a crater - the look of the inside of the crater looks so plateu like. Same with the indented text, in fact I think the top light source makes it stronger, and I did eventually recognise the crater, but I had to look closely at the shadows, and looking at the light side makes it look like a mountain no matter what.
He's attributing way too much importance to the "lit from above" assumption. We are all very used to seeing images shot from above where that doesn't apply. I saw the crater as a plateau in both orientations, and saw the print as indented / raised opposite to what he thought we should.
@LeoDaFinchy Or neurological. I'm autistic.
So, these replies: do the commenters regularly view satellite imagery? Are those images more typically from the northern or southern hemisphere? Do the commenters work in specialist lighting conditions?
I'm sure someone could wring a doctorate out of studying the various ways humans interpret relief patterns and sociological effectors on that.
Yes, same! All my life, many photos of the surfaces of other planets or their moons have looked to me like they’re covered in mountains, not craters.
Yeah it was a mountain for me both directions. Then after reading this comment I went back up and rewound the video to when it was supposed to be in crater mode, and... it was a crater. I let it flip back to mountain mode, and it was still a crater. So for me at least, it seems to get "locked" as how I last saw it, regardless of rotation.
We use similar methods in the game industry to extract height and normal maps using photogrammetry.
Would probably be good for 3d meshes in video game development
This is a gel sight. When you do analyze you can also create a 3d printable image enhancement.
This would be insanely valuable for creating texture depth maps for 3d models in like a video game or something.
My favorite map in Halo 7 is Terrain Designer's Left Eyebrow
Your prints are pretty good :D
Nice layers no gap, no loose belts, nice
I work with lots of 3D LIDAR scans and one huge issue for us is the reflection off of reflective surfaces such as mirrors. Interseting how this gets rid of that, at least on a micro scale.
Mirrors look really interesting in a 3D picture. They're kind of like a hole.
Would training an algorithm be easier using this technique as a source of truth?
Small images, and maybe large images later, could be developed accurately with this, but would that information from this technique help you train a model faster? And further, would information about small things like the quarter or matchhead be useful for larger scale applications of LIDAR such as autonomous vehicles?
Just tape a gel cube to the sensor and drive into things!
As someone who has taken apart watches, I knew exactly what that mystery object was 😂
There was NO way i expected for that to be handheld! Thats NUTS
I think my reaction to your example at 6:00 brings up an interesting point. See, I DIDNT experience the crater illusion when you used the touch microscope, and I think it's because humans are VERY good at context clues and learning; Because you'd shown me other visuals of the touch microscope, I was able to tell from the patterning around the letters that a flat, intended surface was pressed against the microscope, and that the letters were smooth because nothing was touching the gel surface. If they really had been protruding from the block, it would be the letters that had texturing, not the background.
IAs VFX artist I love that it creates normal textures. That´s a totally cool device for scanning hightmaps and normal maps. Especially for fabrics and general surface imperfections. Exciting technology
Coolest technology I've seen in years. Makes you think about what else might be out there and kept from the public.
No money in keeping it from the public.
Nearly every mundane object under that type of imaging looks quite mesmerizing! I actually thought your stubble was one of the more interesting ones. It really shows just how cleanly the blades cut the hairs at the ends. The draping problem was the first thing that came to mind as someone who has done a lot of vacuum forming. But that’s very interesting that it’s not much of an issue as long as they can get the depth value. And as long as it’s not deeper than it is wide.
The "crater" looked like a mountain to me no matter the angle. Even after you said it was a crater and flipped it, my brain couldn't un-mountain it. :D
6:07 "This image looks like it's sticking out because the shadows are on the bottom, but by flipping the light source you see the reality that they're actually indented"
That was the exact opposite for me. The first instance was clearly indented while the second was and emboss.
Very interesting. I can visualize this technology being useful in coin collecting. Replacing an individual's perception on flaws or abnormalities that make a coin valuable, a definitive definition would bring consistency.
Wow this is an interesting use-case, but who would use it? regular collectors or like curating organizations? I don't know the field, so I'm not entirely sure if there's like a central authority organization or the like
as a watch guy int interesting to see a miyota(owned by citizen watch company) quartz movement inside of a casio watch who can and does make their own quartz modules
This is the real life equivalent of a matcap or clay render in 3D rendering.
4:37 That's called monocular depth estimation and it very much already exists. There are quite a few open-source models that can estimate depth fairly well from a single flat image, and if you have multiple images from different angles there are neural networks that can build up an accurate model of the entire scene, sufficient to move a virtual "camera" around and through the scene and create new views in the process.
But how reliable and accurate are they? Probably not good enough for the purposes these camera gel deforming sensing things are bought for?
5:00 Thanks for making the left/right arrangement compatible with the cross-eye technique! 😸👍
this is really interesting! I just learned about vanta-black objects. humans cant exactly figure out what shape vanta black is just by looking at it so it would be nice seeing vanta-black objects being used
Photometric Stereo is the proper name for the algorithm used like Steve said (comes from the original 1980 paper), but the name can be misleading, because it does not use stereo camera system (e.i. a camera pair), only a single camera. It is based upon the better named Shape From Shading algorithm because it reconstructs the depth information from shading, as opposed to camera motion, positional disparity between cameras, etc.
Saying that this microscope uses touch is a bit of a stetch (I think it's arguable either way), this device is mostly optical, but interestingly there ARE microscopes which directly measure by touching the object called scanning probe microscopes. There's a neat video about an atomic force microscope on the Breaking Taps channel.
I've been following this company for many years. Their technology is highly interesting. I'm mostly interested for bio-inspired designs and collecting surface profilometry of scales, scutes, skin, etc.
A disadvantage of the method is that the surface of the gel degrades with use. Perhaps they are trying to improve this design, but I think it is probably unavoidable to some extent. That consumable cost is kind of expensive for casual use, thus restricting this method to only industrial uses.
I ran atomic force microscope (AFM) that used a tiny needle to bounce along the surface and generate images in the nanoscale.
This is honestly amazing! You're so lucky to be able to play with. Is there any information on whether this is going to be sold commercially? I'd love to get my hands on this.
@Barrie Shepherd i mean , that happens with pretty much everything, the Spot robot of Boston Dynamics is 75k meanwhile the chinese knockoffs are way less expensive, even if both were of the same Quality , the original Spot would cost more due to all of the R&D involved , in the other hand the knockoff doesn't has that much R&D into it , and is less expensive due to that
@mustardofdoom Oh man, I guess us common folk won't be able to use it :/
I once received a quote for one about 4 years ago. It is in the tens of thousands to own. They offered a rental arrangement that is in the thousands. I don't know how much the gel pads cost, but they are consumable.
@randallrun i would, if i had half the resources they have
They should pressurize the device to exert pressure on the gel to reduce the draping effect.
This could be an award winning video for the “oddly satisfying” category, i just can’t stop looking at it
06:55 On an aircraft "Traditionally you would create a cast of the scratch and then measure the cast in the lab" - I never saw that. We could use ultrasonic inspection or other forms of non-destructive testing in-situ to measure the depth of the scratch. In any case, best to smooth it out to avoid stress concentration and the risk of crack initiation - re-checking to ensure that the panel thickness is within tolerance or safe limits.
Interesting, the images that the 3d scanner (I think that's maybe more apt than a microscope) puts out already reminded me a lot of normal maps, tech we use in games to cheaply render 3d geometry on a flat polygonal surface. Which actually look exactly like the kind of purple image you see at 2:42 . Funny to see this tech show up in unexpected places
This should be used for handprint-scanning door security access. It'd be hard to fake it. You'd need the actual hand, not just an image of the hand. I love how sci-fi shapes real technology.
Mould is on another level with his science videos. He always finds the most interesting and obscure stuff.
Metrology has to be some of the most fascinating disciplines
Steve. Look up NeRFs! You can do 3d photogrammetry and capture reflective objects. It's a bit different than you described doing but still relevant I think. Unless you mentioned them and I missed it.
when you were explaining the optical illusions of indented and sticking out parts, my brain works exactly opposite for some reason. the ones you called indented looked sticking out to me, and the ones you called sticking out looked indented. interesting...
This would be great for looking at error coins under the microscope !!
Omg this is UN REAL! So many applications. It’s like…an “Approach-a-scope” so sick! 🎉
This would be great for artefact studies in archaeology
@Flavio D. UtzeriSame, I'm doing my bachelor's thesis on teeth abrasion due to nonmasticatory (not chewing) behavoiur, I think the results of such a scan could be useful
@AbydosianChulac2 oh yeah absolutely and it would be far preciser than our eyes
@Mikkel Højbak I was mainly thinking about markings on rustued metal, but on the other hand if you take (for example) a coin, you just scan both sides and you don't have to handle the coin nearly as much, which would make it far less likely to crumble.
@Mikkel Højbak well I can imagine a lot of situation in which force it's not a problem. There are some artifacts, like terra sigillata (I don't remember now the english name, sorry) which may not suffer. Or stoneware
I'm doubtful about the willingness of the archaeologists to press the artifacts into a gel. Things could easily break from the force.
Loving the expensive high-tech microphone mounting system you have set up! Hey it works lol
Apart from the extremely interesting features and applications of this idea, one of my takeaway messages is the nth confirm that Hexagons are the Bestagons 🤓
I’m wondering how big this ‘microscope’ can get. If the depth can’t be more than the width, it would mean bigger microscopes would have more overall use if it were wider/bigger i’d assume
So far my biggest problem with AI, is it seems to be the new fantasy we cling to about salvation. Also for all the problems ai creates the offered solution is often just more ai lol
hi 3d artist/developer: these are SSBump maps
though, more like 2x SSBumps, as SSBs use 3 light sources (packed into the R G and B channels of an image) rather than 6
3 is good enough for realtime rendering
its rather hilarious to see "machine learning" mentioned in this regard because SSBump maps are below dead stupid simple. I do think this device is a *far* better visualization and explanation of how & why SSBump maps work (than the whitepapers ive seen on them anyway) and if i ever need to explain them i might use this video
I saw this (specifically the tiny GelSight mini) recently and thought "what would anyone use this for?". Now that I actually see the accuracy and resolution it has it makes sense
Nice video, I taught they will use ultrasonic imaging medical sensor to see the depth.
Well AIs can already create depth maps just from images. But of course not with the fidelity and resolution of this device. On the other hand you cannot use a gigantic gel pad to create a depth map of a room. :D
Really nive device. Astonishing how many details it still retains 8:43