Tactile 3d serial

Given V as the number of focal points, the rendering frequency f R is defined as shown in equation Equation 21 demonstrates an inversely proportional relationship between the rendering frequency and the number of focal points. This implies that generating higher resolution tactile display requires an increase in the number of focal points to form the tactile object. This in turn results in lower rendering frequency if maximum acoustic pressure per focal point is to be maintained where all transducers contribute to the focal point formation.

On the other hand, in order to increase the rendering frequency of tactile objects, a decrease in the number of focal points per object is to be expected. It then becomes a user choice to pick the proper configurations of the Haptogram system to make specifications for a particular application. Therefore the relationship between the rendering frequency. A larger rendering frequency can still be achieved with the same number of focal points but with lower acoustic pressure.

As long as humans perceive the generated acoustic pressure, such alterations remain valid system configurations. An experiment is conducted to measure the focal point formation delay using one implementation of the Haptogram system. An ultrasound receiver in used to measure the generated acoustic pressure at 15 cm elevation. The performance of the Haptogram system was evaluated to assess the effectiveness of tactile simulation using the Haptogram system.

The analysis focused on the use of a single tile in the Haptogram display. In order to measure the spatial distribution of the acoustic radiation pressure, the experimental setup shown in FIG. An ultrasonic sensor probe was attached to the end effector of the robotic arm ST Robotics R17 whose resolution for movement was 0.

Its output was fed to AC-to-DC conversion circuit to a rectifying bridge and then to an array of decoupling capacitors. A simple script was developed to control the robotic arm to scan an adjustable 3D volume on top of the ultrasound array to measure the distribution of the acoustic radiation forces.

Data were acquired at every 1 mm around the focal point in the case of single point stimulation and 2 mm for 2D and 3D shapes. At a general position, 20 measurements were taken at 1 ms intervals and the average is calculated and recorded as the force actuation at that position. For the single point tactile stimulation scenario, the scan surface was adjusted with the following configuration: 50mm by 50 mm, step size of 1 mm giving a total of measurements for a single slice along the vertical axis.

The robotic arm performed 10 slices of measurements along the vertical axis that are centered at the tactile stimulation point. Since the force amplitude turned out to be time varying, the average values were recorded. The amplitude is normalized with a maximum force of 2. As shown in FIG. Note that side lobes around the main lobe are generated that are nearly symmetrical. Another study was conducted to estimate the optimal elevation of the workspace for one tile of the Haptogram system.

A series of focal points starting from an elevation of 8 cm up to 17 cm were projected and the corresponding force presented at each elevation was measured, with a step size of 1 cm. The results, as shown in FIG. This is justified by the fact that as the elevation increases beyond 14 cm , the ultrasound signals experience higher attenuation, which results in lower perceived forces. On the other hand, as the elevation decreases below the 14 cm height, some transducers would have less contribution towards the formation of the focal point due to limited directivity of the transducers.

Therefore, it would be best to have the center of the device workspace for one tile at the 14 cm elevation since this will produce maximum tactile stimulation, and consequently result in better perception of the displayed tactile stimulus and higher quality of user experience. For the 2D tactile stimulation, two 2D shapes, a straight line and a circle, were considered.

The scan volume was adjusted with the following configuration: mm by mm, step size of 2 mm giving a total of measurements for a single slice along the vertical axis. The robotic arm performed 10 slices of measurements along the vertical axis that are centered at the plane where the 2D shape was generated. Note that side lobes around the main lobe are generated, in both cases, that are nearly symmetrical.

For the 3D tactile stimulation, a display of the upper half of a sphere whose center is located at 14 cm elevation, with a radius of 50 mm was considered. The scan volume was adjusted with the following configuration: mm by mm, step size of 2 mm giving a total of measurements for a single slice making a horizontal plane. The robotic arm performed 20 slices of measurements along the vertical axis that started from height 14 cm up to 24 cm, with a step size of 5 mm between slices a total of 20 slices.

The plot in FIG. A simulation experiment was conducted in order to confirm whether the generated acoustic radiation forces are sufficient for human perception of tactile sensation. The 3D object used in this simulation was the lower half of a sphere located centered mm above the transducer array with a radius of 50 mm. A graphical representation of the hemisphere is shown in FIG.

The resonant frequency was 40 kHz, while the directivity is 80 deg. The following parameters were used to run the simulations described below:. One of ordinary skill in the art will appreciate that a lossless medium is the air interface. If the medium were not lossless i. The frequency e. The frequency is a constant value based on the hardware transducer in use. Beta is the attenuation coefficient of an ultrasonic wave in air interface.

Beta is also a constant. The speed of the ultrasonic wave e. The speed of the ultrasonic wave is constant. P0 is the power density of the ultrasonic signal at the transducers array surface. Voltage e.

Typically, a recommended voltage is provided in the ultrasonic transducer's datasheet. K is the wavenumber of the ultrasonic wave. W is the angular frequency.

Another configuration parameter that can be controlled through the simulation is the resolution of tactile 3D object. Note that higher resolution display takes more time to display the entire 3D object, and thus, require faster hardware to implement. However a low resolution display might degrade the quality of user perception of 3D tactile sensation.

Usability testing will be conducted in the future work to find the optimal tradeoff. In the simulation, the following simulation loop was utilized:. These values were selected to maximize the density of ultrasound transducers in the array.

Other values may be used other researchers have tried random distribution of the transducers spacing. See Gavrilov, L. Next a loop that runs at 1 kHz rate was executed. The loop starts by setting up the coordinate grid to cover the full width of the transducer array in the x direction and to measure the pressure field in the z direction. Then the coordinates of the next point on the 3D model are retried and used to generate a focal point at the desired point and hold on for specific time.

Table 1 shows simulation results for a combination of 16 different configurations with four resolution levels of display and four layout configurations. The forces were calculated by utilizing Equation 8 with a cubic unit of 1 mm 3.

The results of the simulation showed:. The simulation results showed that the proposed system is capable of generating forces that are within comfortable margins of forces that are perceived by the human skin. Furthermore, the three dimensional tactile feedback system is capable of generating a sequence of tactile points that altogether construct a hemisphere in the simulation described above at a 1 kHz update rate.

A similar simulation analysis was conducted to compare the experimental results for various tactile objects with the simulation results. Table 2 summarizes the comparison between experimental and simulation results for four different shapes: a single point, a straight line, a circle and a hemisphere.

For a single point of tactile stimulation, the average tactile force generated is the highest compared to the other objects whereas the standard deviation is the least. This is explained by the fact that producing a single point of tactile stimulation involved minimum interferences especially those due to switching from one focal point to another.

It was observed that 3D objects have the least average tactile stimulation and highest standard deviation since interferences are higher in this case. In all cases, it is clear that all forces simulation or experimental are way above what the human skin can perceive and thus the Haptogram system can be used as a haptic display interface.

The experimental analysis has clearly demonstrated the ability of Haptogram system to generate perceivable tactile objects single point, 2D shapes and 3D shapes. One tile of the Haptogram display was able to generate an average force of 2. The Haptogram system has several advantages as a tactile display. First of all, it does not provide an undesirable tactile feeling originating from the contact between the device and the skin because the device is contactless.

Second, the temporal bandwidth is broad enough to produce 3D tactile feelings by generating a series of tactile focal points and switching between them at a high speed.

The current implementation breaks down any 3D object into a maximum of focal points. An audible sound is heard when the Haptogram system is turned on. The audible sound may be loud such that it can be heard within several meters from the display and it can be annoying for some users. Headphones for ear protection can be used to prevent the user from hearing the audible sound. There are two sources of the audible sound. One is the envelope of the ultrasound. If Hz modulation is used, the Hz audible sound is produced due to the nonlinearity of air, which is a phenomenon utilized in a directive loudspeaker.

The other cause would be the discontinuity of the phases of the driving signals when the position of the focal point is changed. The inventors believe that the former source of the audible sound is the dominant one. There might have been few sources of errors while taking measurements via the robotic arm.

First of all, the surface of the ultrasound array and the scanning surface of the robotic arm may not be perfectly parallel. This implies that there might be some errors measuring forces at a particular horizontal slices due to a skewed measurements.

Secondly, the sound probe attached to the robotic arm is an ultrasound transducer that is not perfectly sensitive to the measurement of forces. Another source of error may have occurred due to additional reflection noises as the robotic arm came closer to the surface of the two-dimensional array of ultrasound transducers.

A usability study was conducted to investigate how well users can perceive animated 2D shapes displayed by the Haptogram system. Although 3D shapes can be presented, the usability study was limited to 2D objects. The shapes were randomly selected and displayed in a square of 10 cm by 10 cm workspace with a precision of 1 mm. Users were asked to feel the tactile stimulation through his or her palm. The users were also given noise cancellation headphones to eliminate acoustic noise generated by the system or any other auditory source that may distract the user from recognizing the tactile shape.

The following configurations were used for tactile stimulation: the rendering frequency was set to 10 Hz, while the number of focal points was set to The shapes stimuli were displayed in a random order to reduce the bias and ordering effects as much as possible. For each trial, the recognition rate and the recognition time were measured.

The recognition rate was defined as the ratio of accurately identifying a displayed shape over the total number of trials. The recognition time was defined as the average time it took the user to recognize a particular shape correctly. The experiment was divided into two blocks of 12 trials for each block, giving participants time to rest from a trial to the next.

A hand-resting stand was designed specifically for this experiment. Active noise-cancelling headphones were used to cancel out the auditory noise. Fifteen 15 adult subjects were participated in the experiment; 8 male, 7 female, average age of 26 years standard deviation was 6. By self-reporting, none of the subjects had any deficiency in their ability to touch. Users were allowed a training session for as much as they desire to familiarize themselves with the system and the stimuli before completing the experiment.

The experimental task required users to recognize one of the four 2D shapes by holding their hands on top of the Haptogram display. No audiovisual cues about the displayed 2D shapes were given to the participants. Each trial started when the participant hit the return button of the keyboard in front of them.

The participant responded by keying a corresponding number 1 for circle, 2 for plus sign, 3 for line, and 4 for triangle. Each shape was displayed continuously for a maximum of 30 seconds. If the participant failed to give a response within the 30 seconds, then the trial was cancelled. As soon as the participant gave a response, the response time was recorded along with whether the selection was correct or not.

Next, the participant was given some time to rest before proceeding to the next trial. The overall recognition rate for all subjects was well above the chance level average of The overall recognition time had an average of Looking at the recognition rate for each shape, the plus sign shape was the easiest to recognize, while the line shape was the most difficult to recognize.

The average recognition rate for all the shapes, along with the standard deviations, are shown in FIG. It was also observed that one participant never got the triangle shape. As for the recognition time, results show that the plus sign shape had the largest recognition time compared to other shapes, while the line shape had the lowest recognition time FIG.

However, the differences are not that significant to derive definite conclusions about the recognition speed for each shape; more or less all shapes have similar recognition times.

In an effort to recognize which shapes were the most confusing, the data was analyzed to find errors across combinations of shapes. Table 3 shows the results e.

The circle was also highly confused with the line Even though it would be hard to derive statistically valid conclusions, a considerable improvement in performance from session to session was observed. This clearly shows that users have quickly learned how to use the device and effectively perceive 2D shapes. The embodiments described above are directed to a Haptogram system capable of displaying 3D tactile objects. The Haptogram system enables users to feel virtual objects without physical touches on any display devices.

The Haptogram system may include one tile having a two-dimensional array of ultrasound transducers e. The ability of the Haptogram system to display various types of objects single point, 2D shapes and 3D shapes was validated in the experiments described above.

As a result, it was confirmed that 1 humans can feel a localized focal point clearly average force generated is 2. If you want to acquire a new measuring system and are not sure whether you want a tactile or optical one, finding out which accuracies are needed is the first thing to do.

A rule of thumb says that the system accuracy of the measuring system should always be increased by a factor of five to ten than the highest required tolerance to be measured. This means: If the tolerance of a characteristic is, for example, 0. In the automotive industry, gears, crankshafts and engine blocks are the classic candidates for tactile measurements: The tolerances and accuracies to be adhered to of these parts require the highest possible degree of precision.

This accuracy is currently barely given with optical measuring systems. What speaks against tactile measurement technology is the high time investment if higher data densities are required: Probing hundreds of measuring points on one workpiece can take a long time, sometimes several hours.

Therefore, a complete check is barely possible in production — because of the time investment and because many CMMs often cannot be placed directly in production. To save time, the number of measuring points can be reduced, but this is at the expense of the data density.

Here, the ratio of time investment and data density always has to be carefully weighed. No matter how many measuring points are captured under highest care: It is not possible to measure the entire surface of the measuring object. This is where the optical measurement technology comes into play: Optical measurement technology is not only faster but also creates a digital image of the entire measuring object and therefore delivers more detailed quality information than tactile measurement technology.

Optical measuring systems e. The measurement is non-contact; the measuring sensor is never touching the surface of the measuring object. The non-contact measuring principle has a decisive advantage on sensitive object surfaces: Damage on the measuring object can be safely prevented. Wear and tear as known from tactile measuring systems also cannot occur. The measuring procedure with optical 3D measuring systems is very simple: The measuring object is placed in front of the sensor — either manually or robot-guided.

Then, the image acquisition begins: The measuring sensor captures step by step every side of the measuring object.

To capture the entire surface, either the workpiece is moved, so that the sensor can capture all areas, or the sensor itself is moved around the workpiece. Then, the connected measuring software automatically transforms all individual measurements into a common coordinate system.

This command is explained further below. This data can be immediately directed into a file if the user wishes to do so. Alternatively, the serial connection can be opened using any programming language or using a Serial Terminal, making it extremely convenient to start using. Sensor axes are defined as follows. This is relevant to understand the output readings that are explained in the next section.

The unit outputs data as text human readable in CSV comma separated format. The format has fixed length and format. It includes a timestamp of when the data was acquired and 3 value coordinates X, Y, Z vectors that can be used to calculate both the direction and value size of the vector of the force being read on each sensor. A typical reading output would look like this: ,,,,15,,,,,,,,,,,,,,.

Column 1: Line type - means a reading OK — Indicates a line with verbose indication after a command is issued. Normally this appears when commands are sent the unit and it responds to them ERR — indicates an error during operation. ERR lines normally appear after sending an invalid user command or at boot, when there is any initialization fault for example a sensor is connected but no calibration data was supplied to the DAQ This column is meant to separate value lines from status lines and simultaneously allow the merging of relevant feedback data in the datastream.

When analyzing data, in a spreadsheet for example, a simple filter on this column would eliminate any undesired lines. Columns 2 and 3: Timestamp The timestamp indicates the moment when Data was acquired for this set of results. Column 2 indicates the seconds and column 3 the milliseconds. Upon boot column 2 shows the number of seconds since the DAQ was powered on.