Improve Open Flexure Microscope imaging

[MatPopp]

related to #141

Motivation:

• for Autonomous battery slurry optimization system #141 it would be nice to measure particle-size distributions, to correlate them to vicosities of mixed slurries, but also to performance of assembled cells.

Current Situation (at AC Training Lab)

• there are some Open Flexure Microscopes, equipped with 40x and 100x Microscope objective lenses.
  The microscopes work well under the following conditions:
• transmission illumination:
  ** requires transparent sample carriers + lamp on top (the light in the office is already enough illumination)
• Sample must be positioned within working distance of microscope, which is below the thickness of a microscopic slide, both for the 40x and 100x objective lenses. -> sample must face downwards towards the objective lens, which excludes transmission illumination in some cases.

Optimization experiments:

##Lightning conditions

Problem:

• I could not adjust the available lightning source (LED -> Pinhole -> Condenser Lens -> tube lens -> objective lens -> sample) to get an evenly illuminated image. I am still not sure, if there is a principle problem with the original setup, or something is wrong with ours (different condenser lens / LED?). The (epi-illumination) Images in https://opg.optica.org/boe/fulltext.cfm?uri=boe-11-5-2447&id=429869 look better than the best I have observed with our setup.

• Video depicting the situation (lines of deposited battery slurry on aluminum foil, with 40x objective lens):

https://github.com/user-attachments/assets/204d6656-df2b-43e0-b9f8-2deaea4dd363
-> the automatic image correction does weird things at the edges, because of the uneven illumination.

Improving illumination:

• I built a 3D-printed illumination-Unit with improved mechanical stability of the condenser-lens mount and an exchangeable screen-Unit:

• Lightpath: LED -> (exchangeable) screen with paper-piece as diffusor -> condenser Lens -> ... (as above)
• reasoning: with the current setup it is super difficult / impossible to actually achive Köhnler illumination https://zeiss-campus.magnet.fsu.edu/articles/basics/kohler.html where the excitation light is focussed to the back-focal plane of the microscope. I could not get rid of bright spots, so I reasoned that homogenizing the light source would also homogenize the image.
  Result: It is still not perfect but a bit more homogeneous than before.
• In theory, options like dark-field mode should be possible with this kind of setup by exchanging the pattern on the screen: (Projecting patterns on the back-focal plane of the objective lens corresponds to choosing angles of illumination. A ring corresponds to only large angles with respect to the optical axis)

• The patterns with rings/blocked center are ment for a dark-field mode. This did not work well, probably due to non-optimized focus on back-focal plane.

Summary:

• Illumination in epi / reflection mode became a bit more homogeneous.
• Different objective lenses require different screen-hole-sizes for optimized homogeneity, they can be exchanged, now.
• The Lens-positioning still needs to be optimized (tunable positioning would be nice)

Improving contrast:

• I could not find a way in the open flexure software to increase contrast of brigth images (necessary, when a lot of back-reflections are present (e.g. in reflection mode) or the contrast of objects to background is just low)
• I wrote a small python - panel app that grabs the image-stream from the open-flexure microscope and increases the contrasst for each color individually.

import cv2
import numpy as np
import panel as pn

cap = cv2.VideoCapture('http://192.168.1.199:5000/api/v2/streams/mjpeg')


def brightness_transform(value, r_low = 100 , r_high=255, g_low=100, g_high=255, b_low=100, b_high=255):
    r_array = np.array(value[:,:,0], dtype=float)
    g_array = np.array(value[:,:,1], dtype=float)
    b_array = np.array(value[:,:,2], dtype=float)

    r_array = (r_array - r_low) * 255 / (r_high - r_low)
    g_array = (g_array - g_low) * 255 / (g_high - g_low)
    b_array = (b_array - b_low) * 255 / (b_high - b_low)

    r_array = np.where(r_array < 0, 255, r_array)
    r_array = np.where(r_array > 255, 0, r_array)
    g_array = np.where(g_array < 0, 255, g_array)
    g_array = np.where(g_array > 255, 0, g_array)
    b_array = np.where(b_array < 0, 255, b_array)
    b_array = np.where(b_array > 255, 0, b_array)

    value = np.stack([r_array, g_array, b_array], axis=2)

    return value

## a panel with sliders to adjust low and high values:

r_slider = pn.widgets.RangeSlider(
    name='blue', start=0, end=255, value=(0, 255), step=0.1)
g_slider = pn.widgets.RangeSlider(
    name='blue', start=0, end=255, value=(0, 255), step=0.1)
b_slider = pn.widgets.RangeSlider(
    name='red', start=0, end=255, value=(0, 255), step=0.1)

def autocalibrate(event):
    ret, frame = cap.read()
    frame = np.array(frame, dtype=float)
    r_low = np.min(frame[:,:,0])
    r_high = np.max(frame[:,:,0])
    g_low = np.min(frame[:,:,1])
    g_high = np.max(frame[:,:,1])
    b_low = np.min(frame[:,:,2])
    b_high = np.max(frame[:,:,2])
    r_slider.value = (r_low, r_high)
    g_slider.value = (g_low, g_high)
    b_slider.value = (b_low, b_high)

autocalibrate_button = pn.widgets.Button(name='Autocalibrate')
autocalibrate_button.on_click(autocalibrate)

slider_col = pn.Column(r_slider, g_slider, b_slider, autocalibrate_button)

pn.serve(slider_col, threaded=True)

while True:
    ret, frame = cap.read()
    print("max of frame", np.max(frame))
    r_low, r_high = r_slider.value
    g_low, g_high = g_slider.value
    b_low, b_high = b_slider.value
    transformed = brightness_transform(np.array(frame,dtype=float),
                                       r_low = r_low,
                                       r_high = r_high,
                                       b_low = b_low,
                                       b_high = b_high,
                                       g_low = g_low,
                                       g_high = g_high
                                       ).astype(np.uint8)
    print("max of transformed", np.max(transformed))
    cv2.imshow('frame', frame)
    cv2.imshow('transformed', transformed)

    if cv2.waitKey(1) == 27:
        break
cv2.destroyAllWindows()

Examples of the App in action are shown below.

Comparison between objective Lenses:

There were 3 Lenses available:

• 100x oil immersion lens, 160 mm correction (?), working distance WD < 0.5 mm (?)
• 40x , 160 mm correciton, working distance < 0.5 mm
• 20x, infinity corrected , working distance 8.4 mm. (an adapter was 3D-printed for the M27 -> RMS thread (from openflexure)), as well as a platform to put samples into the WD.

I took some images of (water) drop-casted NMC on various substrates:

NMC on white 3d-printed PLA

• 100x objective lens
  
  -> poor image quality in general. The surface of the 3d-printed carrier is too uneven + generally poor contrast.

• 40x objective lens
  
  -> enhanced contrast image not good but useful

• 20x objective lens,

-> the lines in the images should correspond to 0.4 mm nozle diameter.
-> wider field of view in trade for less details within particles.
-> the contrast falls off towards the edges of the image. Probably due to infinity correction (open flexure is built for 160x)

NMC on microscopic slide (-> transmission illumination)

• 100x objective lens, sample facing towards lens:
  
  -> optical blurr >> 1 pixel. probably since oil immersion lens is used with air gap
  -> focus and sample difficult to find.

• 40x objective lens, sample facing towards lens:

-> good contrast already in original image (forgot to enhance contrast manually)
-> probably preferrable magnification for this use-case
-> only image where all componets are used as intended by manufacturer.

• 20x objective lens, sample facing towards lens:

-> good contrast, again sharp in center but blurry at edges

• 20x objective lens, sample facing away from lens (image taken through microscopic slide):

-> only very slightly deteriorated contrast, that can be corrected for (due to reflection of extra surface in beam-path)
-> image quality essentially as well as above. (correction collar was used, but I could not spot a difference except for working distance on this level of quality)

Summary for (battery material usecase):

• For evaluation of particle size distribution, dropcasting of a water-dispersed NMC material might be applicable.
• The high WD of the 20x objective is preferrable
  • to be able watch samples through object carriers / bottom of wellplates
  • reduced risk for crashes
• For a good mix of detail and still good field of view, 40x objective is preferrable.