algos

deepdataspace.algos

This module provides kinds of algorithms for processing dataset.

calculate_fnfp

deepdataspace.algos.calculate_fnfp

Compare predictions to ground truths, found the FPs and FNs.

calculate_iou(all_gt: ndarray, all_det: ndarray) → ndarray

For every ground truth, calculate it’s iou to every prediction.

Parameters:

• all_gt – (np.ndarray) Shape (G, 4), 4 present [x1, y1, x2, y2]

• all_det – (np.ndarray) Shape (D, 4), 4 present [x1, y1, x2, y2]

Return iou:

(np.ndarray) Shape (G, D)

calculate_thresholds(all_gt: List[List], all_det: List[List], iou_thresh: float = 0.5) → List[Dict[str, float]]

For given IoU thresh, calculate confidence thresh for precisions from 0.0 to 1.0 .

Parameters:

• all_gt –

  All ground truth objects from a subset.

  [
      # [image_id, category_id, [x1, y1, x2, y2]]
      [1, 1, [10, 20, 30, 40]],
  ]
  

• all_det –

  All prediction objects from a subset.

  [
      # [image_id, category_id, [x1, y1, x2, y2], conf]
      [1, 1, [10, 20, 33, 43], 0.8],
  ]
  

• iou_thresh – float.

Return conf thresholds:

list of dict.

[
    {
        "conf_thresh": 10,
        "recall": 10,
        "precision": 10,
        "precision_thresh": 10.1
    }
]

calculate_fnfp(all_gt: List[List], all_det: List[List], iou_thresh: float = 0.5) → Tuple[List[int], List[int]]

For given IoU thresh, check the correctness of all predictions in an image.

Parameters:

• all_gt –

  All ground truth objects from a subset

  [category_id(int), bbox(List[int])], bbox = [x1, y1, x2, y2]

• all_det –

  All prediction objects from a subset

  [category_id(int), bbox(List[int]), conf(float)], bbox = [x1, y1, x2, y2]

• iou_thresh – IoU threshold

Return tuple of list of int:

(gt_results, det_results)

gt_results, list, -1 means FN, otherwise means matched det id

det_results, list, 1 means TP, 0 means FP

graph

deepdataspace.algos.graph

This file implements the graph data structure.

class Graph(num_nodes)

Bases: object

This implements a symple non-directed graph

property num_nodes

add_edge(v: int, w: int)

neighbor_nodes(v: int)

has_circle()

topology_order()

class DirectedGraph(num_nodes)

Bases: Graph

This implements a symple directed graph

add_edge(v: int, w: int)

class SymbolGraph(num_nodes)

Bases: Graph

This implements a symple non-directed symbol graph, whose nodes are symbols instead of int numbers.

add_node(v: Hashable)

add_edge(v: Hashable, w: Hashable)

neighbor_nodes(v: Hashable)

topology_order()

class DirectedSymbolGraph(num_nodes)

Bases: SymbolGraph, DirectedGraph

This implements a symple directed symbol graph, whose nodes are symbols instead of int numbers.

test()

test2()

refine_by_seed

deepdataspace.algos.refine_by_seed

This module provides a algorithm for resorting dataset by seed images.

norm_embedding(embeddings, axis=0, keepdims=True)

Parameters:

• embeddings – np.ndarray of embedding matrix

• axis – the axis normalized

• keepdims – if True return array will keepdims

Return merge_dist:

1D-np.ndarray, similarity combine pos_dist and neg_dist

calculate_mean_similarity(seed_embd, embeddings)

Parameters:

• seed_embd – np.ndarray of seed images embedding matrix

• embeddings – np.ndarray of all images embedding matrix

Return similarity_dist:

np.ndarray of similarity of seed_embd and embeddings

calculate_min_similarity(seed_embd, embeddings)

Parameters:

• seed_embd – np.ndarray of seed images embedding matrix

• embeddings – np.ndarray of all images embedding matrix

Return similarity_dist:

np.ndarray of similarity of seed_embd and embeddings

merge_dist(pos_dist, neg_dist)

Parameters:

• pos_dist – 1D-np.ndarray, similarity between positive seeds embd and all images embd

• neg_dist – 1D-np.ndarray, similarity between negtive seeds embd and all images embd

Return merge_dist:

1D-np.ndarray, similarity combine pos_dist and neg_dist

similarity_classifier(pos_seeds, neg_seeds, embeddings)

Parameters:

• pos_seeds – a list of positive seed idx

• neg_seeds – a list of negative seed idx

• embeddings – np.ndarray of all images embedding matrix

Return idx_list:

a list of refined and re-sorted idx

train_svm_model(pos_seed_embd, neg_seed_embd)

Parameters:

• pos_seed_embd – np.ndarray of pos seed images embedding matrix

• neg_seed_embd – np.ndarray of neg seed images embedding matrix

Return model:

a trained svc model

svm_classifier(pos_seeds, neg_seeds, embeddings)

Parameters:

• pos_seeds – a list of positive seed idx

• neg_seeds – a list of negative seed idx

• embeddings – np.ndarray of all images embedding matrix

Return idx_list:

a list of refined and re-sorted idx

refine(pos_seeds: list, neg_seeds: list, embeddings: ndarray)

Parameters:

• pos_seeds – a list of positive seed id

• neg_seeds – a list of negative seed id

• embeddings – a np.ndarray of all images embedding matrix

Return idx_list:

a list of refined and re-sorted idx