Architecture features of the mmengine

[morkovka1337]

Hello everyone.

Thanks to developers of openmmlab, we really like your projects!
In my team we are considering to follow the architecture style of mm-like repositories. We will create a lib for a specific task (concretely, 3d tracking based on camera, lidar and radar data), and now I'm researching the features of mm-like repositories we need to implement in our solution.
Previously, every repo was based on the mmcv, but I've noticed that every project is migrating to mmengine now. Therefore I have some questions:

1. (general) What is the intention of the mmengine itself? Why do we need mmengine if we already have mmcv?
2. (specific) What is the main difference with mmcv? As far as I've understood, the functional part of the mmengine and mmcv are partially overlapping, but there are some parts that present in the mmcv and do not present in the mmengine and vice versa. Should we completely reject mmcv in favor of mmengine or should we combine them together?

[HAOCHENYE]

So sorry for my late reply. I've not set up automatic notifications for discussions, so I miss notice this topic up to now.

This is a very good question and I'm sure many people have doubts about it. I hope the following answer can provide you with some reasonable reference.

Over the past few years, OpenMMLab has open-sourced more than 20 algorithm libraries based on MMCV, while also recognizing some limitations of MMCV, such as:

1. There is a lot of redundant code when constructing the Runner in different algorithm libraries.
2. The Runner design lacks flexibility, requiring the Runner to be reloaded for even minor changes to the process.
3. The model update process is not uniform in different algorithm libraries, with some using Hooks to update and some updating within the model.
4. Evaluation is coupled with the dataset, making it impossible to freely combine evaluation methods and datasets.
   ...

In order to address these issues, we have redesigned the new training architecture MMEngine for OpenMMLab and plan to migrate downstream algorithm libraries to MMEngine. Therefore, we strongly recommend that you develop new projects based on MMEngine. We will continue to iteratively update MMEngine, adding more cool features to it, while old versions of MMCV will mainly focus on bug fixes.

BTW, MMCV 2.0 is no longer responsible for constructing the training process and has removed a series of modules related to Runner. When developing based on MMEngine, it is recommended that you consider MMCV as a foundational library for computer vision research that provides a large number of deep learning operators(ops) and data enhancement strategies(transforms).

[morkovka1337]

Thanks for such a detailed answer. I've been workinig on integrating some external model into the mmdetection and based my work on 3.x branch which is based on mmengine itself already. After I have understood some concepts, I was able to easily implement missing modules and put everything alongside with mmdetection as submodule so that I will easily rebase it when it is updated in the future.
When I will need to implement some other mm-like repositories (say, 3d lidar tracking), I will base my work on mmengine.

Thank all the developers for such a great library!