Creating a 2D geometry model from an image poses challenges for CAD users due to factors such as noise, segmentation difficulties, complex geometric structures, scale and perspective variations, and the need for CAD system compatibility. In this paper, we propose a novel deep learning approach utilizing Long-Short Term Memory (LSTM) to address these challenges. Our approach decomposes the shapes in the images into line and curve segments and accurately locates their intersection points. To enhance the model’s performance, we introduce two distinct types of features (angle and curvature features) and optimize the model through hyperparameter tuning. The resulting model exhibits robustness against noise, varying image sizes, and can effectively locate different types of intersection points. To evaluate the proposed model, we have developed a Python-based software and conducted experiments on a dataset comprising of 200 shapes with seven different resolutions. Comparative analysis against a state-of-the- art method (TCVD) from the literature demonstrates that our approach achieves higher accuracy in terms of line, curve, and intersection point detection.

Sketching plays a vital role in the design culture and is an integral part of the design process [

CAD users who contemplate switching between different CAD software often worry about leaving their legacy data behind. The fear of having to remodel everything in a new environment is understandable and a common concern in the CAD practice. The conversion of legacy data is viewed as a significant obstacle to changing software packages, particularly when there is a need to continue ongoing projects or incorporate components from previous designs into new ones, requiring modifications to fit both old and new designs. Dealing with legacy products that may lack available drawings further complicates matters, leaving customers with no choice but to provide a picture and a few measurements to create replacements or upgrades.

Given the crucial role of sketches in the design process, it is highly valuable to optimize sketch creation methods to expedite the overall design process. By providing a tool that can analyze an image and extract the contained information to construct 2D sketches, we can significantly accelerate the concept-to-design workflow.

The detection of corner and tangent points is an active research area within stroke segmentation [

A corner refers to the point where two edges intersect, such as the intersection of two lines within polylines (

There are two major approaches for Corner Detection: Traditional segmentation techniques, which typically rely on heuristic algorithms and empirically determined parameters, like curvature, and machine learning (ML)-based stroke segmentation methods, which use various shape features to train a model. The latter is general and extensible compared to heuristic-based approaches [

As an example of the first approach, Wolin

Xiong and LaViola [

Albert

For the ML-based corner point detection approach, Herold and Stahovich [

Long [

In this study, we use a bi-directional long short-term memory (bi-LSTM) neural network to classify corners, lines, and arcs in the design images. 200 graphical shapes with 7 different sizes containing lines and arcs are created. The border points of the images are calculated using image processing techniques, and then the desired features are extracted. Then, we tested the bi-LSTM model using 10-fold cross-validation. Our results show that our bi-LSTM classifier outperforms existing ML-based classifiers.

We use bi-directional long short-term memory (bi-LSTM) neural network to perform the classification and classify shapes into corners, lines, and arcs. LSTMs are types of Recurrent Neural Networks (RNNs) that use gating functions in their architecture [

The problem of vanishing gradients in ordinary RNNs is solved by LSTMs, which are capable of learning long-term dependencies. LSTMs use hidden layer units as memory cells which remember key information and forget unnecessary ones. _{t} to an LSTM layer. H_{t} and C_{t} are the hidden state and cell state at the time

The original LSTM can only use data from previous steps. To overcome this drawback, the bi-LSTM has been introduced [

For this study, 200 shapes with different resolutions were created using Adobe Illustrator to train and test the proposed method. The shapes contain lines and arcs with different orientations and curvatures. All different types of corner points are introduced in the samples (

To study the LSTM model, a data set that includes 200 unique images with 7 different sizes has been used. Each image consists of a single shape. Each shape can include several strokes (lines and arcs). We used OpenCV library to extract the output contour of each shape. OpenCV (Open Source Computer Vision Library) is an open-source software library that provides a comprehensive set of tools and functions for computer vision and image processing tasks [

The raw sequence of border points is not suitable to be used as features since they don’t explicitly show the geometrical changes in the shape to represent different labels. Here, we used two different sets of features, namely angle and curvature features, which are introduced as follows.

To create both angle and curvature features,

For angle features, the vector direction from point

Then the angle feature between these two vectors is calculated as

We create curvature features as follows: for each point

Then the analytical curvature of the fitted polynomial is calculated.

Feature hyperparameters tuning includes numbers of neighbor counts and values for both the angle and curvature features,

Model hyperparameter tuning, which includes Bi-LSTM initial learning rate, numbers of hidden layers and different class weights,

And after training and during the post-processing, the region with less than specific numbers of certain classes will be marked, and the region will get a label of its neighbor of the left or right regions with these rules:

For a line or curve candidate region (with a smaller size than a certain threshold), if the neighbor labels are different, the candidate region will be converted to a corner, but if the neighbor labels have the same label, the candidate region will be converted to the neighbor label.

For a corner candidate region (with a smaller size than a certain threshold), if the neighbor labels are different, the candidate region will remain the corner, but if the neighbor regions have the same label, the candidate region will be converted to the neighbor label.

We added these two thresholds to our training process to find their optimum for each label. 162 different parameters combinations are identified. Then, using grid search and 10-fold cross-validation, the optimum set of parameters was discovered. Using these sets of parameters, the final model is established.

The original LSTM can only use data from previous steps. The initial results from the LSTM models show that there are plenty of tangent points that are missed. For improving tangent point detection performance, we used the Bi-LSTM model. The Bi-LSTM models show a great improvement in the case of tangent points detection (

To illustrate the difference between the angle feature and curvature feature, we choose a sample shape (

Here, we explore the effect of the number of neighbors in model classification.

As mentioned in section 2.2, 200 shapes with 7 different resolutions are created and tested by our deep learning model. Our database contains 1174 corners, 794 lines and 379 curves. 150 shapes have been used in 10-fold cross validation to find the best hyper parameters and create the final model then the final model is applied on remaining 50 shapes.

Accuracy results for corner detection | TCVD | Bi-LSTM |
---|---|---|

False positive corners | 149 | 53 |

False negative corners | 81 | 49 |

Correct corners found | 1092 | 1124 |

Total corners | 1173 | 1173 |

Correct corners accuracy | 0.93 | 0.96 |

Accuracy results for line detection | TCVD | Bi-LSTM |
---|---|---|

False positive lines | 63 | 19 |

False negative lines | 10 | 6 |

Correct lines found | 784 | 788 |

Total lines | 794 | 794 |

Correct lines accuracy | 0.98 | 0.99 |

Accuracy results for curve detection | TCVD | Bi-LSTM |
---|---|---|

False positive curves | 54 | 25 |

False negative curves | 57 | 24 |

Correct curves found | 322 | 355 |

Total curves | 379 | 379 |

Correct curve accuracy | 0.84 | 0.93 |

For evaluation purposes we define accuracy, false positive rate, and false negative rate as follows:

When evaluating false positive corners, our Bi-LSTM algorithm demonstrated a reduction in the number of incorrect corner detections compared to the TCVD algorithm (

The accuracy results for line detection on 2D shapes using our proposed algorithm (Bi-LSTM) and TCVD are summarized in

Regarding false positive curves, our Bi-LSTM algorithm demonstrated better performance than TCVD by significantly reducing the number of incorrect curve detections (

The experimental findings indicate that Bi-LSTM exhibits better performance compared to TCVD when applied to the tested shapes. It is worth noting that TCVD was primarily designed and evaluated for handwritten strokes. In contrast, our Bi-LSTM model leverages two distinct types of features and incorporates varying numbers of neighbors, which effectively reduces the false positive rate. Additionally, by selecting Bi-LSTM as the deep learning model, which inherently captures both backward and forward information, we observe a notable decrease in the false negative rate, as demonstrated in section 3.1 of our study. These results emphasize the advantages of employing Bi-LSTM in curve detection and highlight its ability to surpass the performance of TCVD on the given shapes.

In this paper, we propose a novel deep learning model that effectively breaks down CAD shapes into constructing line and curve segments while accurately identifying intersection points. Our approach introduces a sequential methodology fed into a bi-LSTM model, showcasing the strength of Bi-LSTM in shape segmentation. Leveraging the backward and featured capabilities of Bi-LSTM, along with the incorporation of two types of angle and curvature features, our model achieves the detection of various intersection point types.

To enhance the algorithm’s robustness to noise and enable segmentation across different scales, we introduce the utilization of different neighbor sizes for feature calculation. The experimental results demonstrate the high accuracy of our model, achieving 96% accuracy in detecting intersection points and 97% accuracy in classifying line and curve segments. These results are compared against the heuristic TCVD model, highlighting the accuracy improvement our approach brings to shape segmentation.

The proposed algorithm utilizes machine learning techniques to detect and classify different shapes based on border points. This can be used in CAD software to automatically identify and label shapes within a design. It saves time and effort by automating the shape detection process, especially when dealing with complex designs that contain numerous shapes.