[BUG]: cannot get real type in python code

[ChunelFeng]

Required prerequisites

• Make sure you've read the documentation. Your issue may be addressed there.
• Search the issue tracker and Discussions to verify that this hasn't already been reported. +1 or comment there if it has.
• Consider asking first in the Gitter chat room or in a Discussion.

What version (or hash if on master) of pybind11 are you using?

2.4.3-2build2

Problem description

how can i change x to MyGParam type in python code?

class MyGParam(GParam):
    def __init__(self):
        super().__init__()
        self.tmp_val = 1

main:
    p = MyGParam()
    createGParam(p, "key")    # input MyGParam() type value

    x = getGParam("key")
    # how can i change x to MyGParam type?
    print(x.tmp_val)    # crash, type(x) is GParam, not MyGParam, so no x.tmp_val

class CppClz {
    CStatus __createGParam_4py(GParamPtr param, const std::string& key) {
        infos.insert(pair<string, GParamPtr>(key, param))
    }

    GParamPtr __getGParam_4py(const std::string& key) {
        auto iter = infos.find(key);
        return iter.second;
    }

    map<string, GParam*> infos;
}

Reproducible example code

class MyGParam(GParam):
    def __init__(self):
        super().__init__()
        self.tmp_val = 1

main:
    p = MyGParam()
    createGParam(p, "key")    # input MyGParam() type value

    x = getGParam("key")
    # how can i change x to MyGParam type?
    print(x.tmp_val)    # crash, type(x) is GParam, not MyGParam, so no x.tmp_val




class CppClz {
    CStatus __createGParam_4py(GParamPtr param, const std::string& key) {
        infos.insert(pair<string, GParamPtr>(key, param))
    }

    GParamPtr __getGParam_4py(const std::string& key) {
        auto iter = infos.find(key);
        return iter.second;
    }

    map<string, GParam*> infos;
}

Is this a regression? Put the last known working version here if it is.

Not a regression

[XuehaiPan]

I once faced the same problem, accessing classes created from the Python side in C++ code. But this seems impossible without using circular references. Because the Python class are created after the C extension is imported.

Here are my workarounds:

Solution 1: Use py::module_::import

// C++ side

class CppClz {
    using GParamPtr = std::shared_ptr<GParam>;

    CStatus __createGParam_4py(GParamPtr param, const std::string& key) {
        infos.insert(pair<string, GParamPtr>(key, param))
    }

    GParamPtr __getGParam_4py(const std::string& key) {
        PYBIND11_CONSTINIT static py::gil_safe_call_once_and_store<py::object> storage;
        auto MyGParam = storage
            .call_once_and_store_result([]() -> py::object {
                return py::getattr(
                    py::module_::import("somepackage.module.to.my_gparam"),
                    "MyGParam");
            })
        .get_stored();
        auto iter = infos.find(key);
        return Convert2MyGParam(iter.second);
    }

    map<std::string, GParamPtr> infos;
}

Solution 2: assign a reference of MyGParam to the C extension in Python code

# Python side

from . import _C_extension

class MyGParam(_C_extension.GParam):
    def __init__(self):
        super().__init__()
        self.tmp_val = 1


setattr(_C_extension, "MyGParam", MyGParam)

// C++ side

py::module_ GetCxxModule(const std::optional<py::module_>& module) {
    PYBIND11_CONSTINIT static py::gil_safe_call_once_and_store<py::module_> storage;
    return storage
        .call_once_and_store_result([&module]() -> py::module_ {
            EXPECT_TRUE(module, "The module must be provided.");
            return *module;
        })
        .get_stored();
}

void BuildModule(py::module_& mod) {
    GetCxxModule(mod);
    
    // class definitions
    py::class_<CppClz>(...)
}

PYBIND11_MODULE(_C_extension, mod) { optree::BuildModule(mod); }

// C++ side

class CppClz {
    using GParamPtr = std::shared_ptr<GParam>;

    CStatus __createGParam_4py(GParamPtr param, const std::string& key) {
        infos.insert(pair<string, GParamPtr>(key, param))
    }

    GParamPtr __getGParam_4py(const std::string& key) {
        auto MyGParam = GetCxxModule().attr("MyGParam");
        auto iter = infos.find(key);
        return Convert2MyGParam(iter.second);
    }

    map<std::string, GParamPtr> infos;
}

[ChunelFeng]

hi, thank you for your help,

i try to use cpp to write a MyGParamCpp(as GParam's son class), and export in pybind11,
than it works well.

// define a param type in cpp
class MyGParamCpp : public GParam {
public:
    explicit MyGParamCpp() : GParam() {};
    ~MyGParamCpp() override {};

    CStatus setup() override {
        // can enter the virtual function in right way
        printf("enter setup .. \n");
        return CStatus();
    }  

    CVoid reset(const CStatus& curSts) override {
        printf("enter reset .. \n");
        return;
    }

    int prm1 {1};
    int prm2 {2};
};

py::class_<GParam, PywGParam, std::unique_ptr<GParam, py::nodelete> >(m, "GParam")
    .def(py::init<>());

// export MyGParamCpp type and its attr
py::class_<MyGParamCpp, GParam, std::unique_ptr<MyGParamCpp, py::nodelete> >(m, "GParamCpp")
     .def(py::init<>())
     .def_readwrite("prm1", &MyGParamCpp::prm1)
     .def_readwrite("prm2", &MyGParamCpp::prm2)
     ;

class MyGNode(GNode):
    def __init__(self):
        super().__init__()
        print("new")

    def init(self):
        p = GParamCpp()    # create a GParamCpp type info
        p.prm1 = 20
        sts = self.createGParam(p, "key")
        return sts

    def run(self):
        print("[{0}] run {1}".format(datetime.now(), self.getName()))
        time.sleep(0.1)
        p = self.getGParam("key")
        print('origin prm1 is ', p.prm1)
        p.prm1 += 10    # ok, can get p.prm1 and the value is right
        return CStatus()

but all the 3 motheds may be a little harder than i first think, such as:

# only use this, with no cpp code. haha, it seems impossible.
class MyGParam(GParam):
    def __init__(self):
        super().__init__()
        self.tmp_val = 1