Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

cv ext:list -L returns incorrect results on WordPress #46

Open
MegaphoneJon opened this issue Dec 13, 2018 · 4 comments
Open

cv ext:list -L returns incorrect results on WordPress #46

MegaphoneJon opened this issue Dec 13, 2018 · 4 comments

Comments

@MegaphoneJon
Copy link
Contributor

This seems to be a problem with the core caching class, but I was unable to trigger this problem except through cv so I'm filing it here.

Using cv ext:list -L on WordPress only returns the extensions that ship with core (iats and api4). The problem seems to be in this line, because \CRM_Extension_System::singleton()->getFullContainer() returns a bad cache. Below is the output from placing print_r($containers['default']->cache); at this line. Note how valueCache['default'] is empty under WordPress:

WordPress (incorrect)

CRM_Utils_Cache_SqlGroup Object                                                                                                                                                                          [80/20072]
(
    [group:protected] => b53cf5f32d1aff8d90a464dd69f7066f
    [componentID:protected] =>
    [valueCache:protected] => Array                      
        (                                
            [civiroot] => Array
                (
                    [org.civicrm.api4] => /ext/api4
                    [com.iatspayments.civicrm] => /ext/iatspayments
                )                                        
                                         
            [default] => Array
                (
                )                
                         
            [full] => Array                           
                (                        
                    [org.civicrm.api4] => civiroot
                    [com.iatspayments.civicrm] => civiroot
                )                
                         
            [mapper_moduleFiles] => Array                   
                (                        
                    [0] => Array
                        (
                            [prefix] => customrenewalstartdate
                            [filePath] =>
                        )                         
                                         
                    [1] => Array
                        (
                            [prefix] => originaljoindate
                            [filePath] =>
                        )                        
                                         
                    [2] => Array
                        (
                            [prefix] => hidepaypalexpresscheckout
                            [filePath] =>
                        )

                    [3] => Array     
                        (
                            [prefix] => membershipcards
                            [filePath] =>
                        )       
                                              
                    [4] => Array
                        (
                            [prefix] => noverwrite
                            [filePath] =>
                        )                                                                                                                                                                                         

                    [5] => Array                                                                                                                                                                                  
                        (                                                                                                                                                                                         
                            [prefix] => wpcontactupdate
                            [filePath] =>                                                                                                                                                                         
                        )                                                                                                                                                                                         

                    [6] => Array                                                                                                                                                                                  
                        (                                                                                                                                                                                         
                            [prefix] => publicmailings
                            [filePath] =>                                                                                                                                                                         
                        )                                                                                                                                                                                                                                                          
                    [7] => Array
                        (                                                                                                                                                                                                                      [prefix] => giftmemberships    
                            [filePath] =>                                                                                                                                                                                                  )                                           

                    [8] => Array                                                                                                                                                                                                           (                                      
                            [prefix] => presspassextension                          
                            [filePath] =>                                           
                        )                                                           
                                                                                    
                    [9] => Array                                                    
                        (                                                           
                            [prefix] => recordfailedpaypalrecur
                            [filePath] =>                                                                                                                                                                         
                        )

                    [10] => Array
                        (
                            [prefix] => csshelper
                            [filePath] =>
                        )

                    [11] => Array
                        (
                            [prefix] => notificationlog
                            [filePath] =>
                        )

                    [12] => Array
                        (
                            [prefix] => failedpaymentsearch
                            [filePath] =>
                        )

                    [13] => Array
                        (
                            [prefix] => reporterror
                            [filePath] =>
                        )

                    [14] => Array
                        (
                            [prefix] => extendedreport
                            [filePath] =>
                        )

                    [15] => Array
                        (
                            [prefix] => membershippayment
                            [filePath] =>
                        )

                    [16] => Array
                        (
                            [prefix] => multiplerecurring
                            [filePath] =>
                        )

                    [17] => Array
                        (
                            [prefix] => innodbtriggers
                            [filePath] =>
                        )

                    [18] => Array
                        (
                            [prefix] => defaultcountrysearch
                            [filePath] =>
                        )

                    [19] => Array
                        (
                            [prefix] => monitoring
                            [filePath] =>
                        )

                    [20] => Array
                        (
                            [prefix] => csvimport
                            [filePath] =>
                        )

                )

        )

    [expiresCache:protected] => Array
        (
            [civiroot] => 1544740853
            [default] => 1544740853
            [full] => 1544740853
            [mapper_moduleFiles] => 1544740855
        )

    [table:protected] => civicrm_cache
)

Drupal (correct)

CRM_Utils_Cache_SqlGroup Object                                                                                                                                                                            [61/984]
(                                                                                                                                                                                                                 
    [group:protected] => d137abb97b85e36aa1f2c08fe1b044a1
    [componentID:protected] =>
    [valueCache:protected] => Array
        (                
            [civiroot] => Array                           
                (                                                                                                                                                                                                 
                    [org.civicrm.api4] => /ext/api4
                    [com.iatspayments.civicrm] => /ext/iatspayments
                )                
                         
            [default] => Array                    
                (                                                                                                                                                                                                 
                    [nz.co.fuzion.extendedreport] => /nz.co.fuzion.extendedreport
                    [eu.tttp.noverwrite] => /eu.tttp.noverwrite
                    [org.civicrm.shoreditch] => /org.civicrm.shoreditch
                    [com.drastikbydesign.stripe] => /com.drastikbydesign.stripe
                    [org.woolman.genderselfidentify] => /org.woolman.genderselfidentify
                    [org.takethestreets.campaignincontributiontab] => /org.takethestreets.campaignincontributiontab
                    [uk.co.vedaconsulting.mosaico] => /uk.co.vedaconsulting.mosaico
                    [org.civicrm.module.cividiscount] => /org.civicrm.module.cividiscount
                    [net.ourpowerbase.sumfields] => /net.ourpowerbase.sumfields
                    [com.aghstrategies.zombiefinder] => /com.aghstrategies.zombiefinder
                    [org.lavalove.emergencycontact] => /org.lavalove.emergencycontact
                    [org.civicrm.flexmailer] => /org.civicrm.flexmailer
                    [com.megaphonetech.monitoring] => /com.megaphonetech.monitoring
                )
                                      
            [full] => Array
                (                                                                                                                                                                                                 
                    [org.civicrm.api4] => civiroot
                    [com.iatspayments.civicrm] => civiroot                                                                                                                                                        
                    [nz.co.fuzion.extendedreport] => default                                                                                                                                                      
                    [eu.tttp.noverwrite] => default
                    [org.civicrm.shoreditch] => default                                                                                                                                                           
                    [com.drastikbydesign.stripe] => default                                                                                                                                                       
                    [org.woolman.genderselfidentify] => default
                    [org.takethestreets.campaignincontributiontab] => default                                                                                                                                     
                    [uk.co.vedaconsulting.mosaico] => default                                                                                                                                                     
                    [org.civicrm.module.cividiscount] => default
                    [net.ourpowerbase.sumfields] => default                                                                                                                                                       
                    [com.aghstrategies.zombiefinder] => default                                                                                                                                                   
                    [org.lavalove.emergencycontact] => default                                                                                                                                                    
                    [org.civicrm.flexmailer] => default                                                                                                                                                           
                    [com.megaphonetech.monitoring] => default                                                                                                                                                     
                )                                                                                                                                                                                                 
                                                                                                                                                                                                                  
            [mapper_moduleFiles] => Array                                                                                                                                                                         
                (                                                                                                                                                                                                 
                    [0] => Array                                                                                                                                                                                  
                        (                                                                                                                                                                                         
                            [prefix] => campaignincontributiontab                                                                                                                                                 
                            [filePath] => /home/jon/local/lava/htdocs/sites/all/civicrm/extensions/org.takethestreets.campaignincontributiontab/campaignincontributiontab.php                                     
                        )                                                                                                                                                                                         
                                                                                                                                                                                                                  
                    [1] => Array                                                                                                                                                                                  
                        (                                                                                                                                                                                         
                            [prefix] => emergencycontact                                                                                                                                                          
                            [filePath] => /home/jon/local/lava/htdocs/sites/all/civicrm/extensions/org.lavalove.emergencycontact/emergencycontact.php                                                             
                        )                                                                                                                                                                                         
                                                            
                    [2] => Array                                                                                                                                                                                  
                        (
                            [prefix] => extendedreport
                            [filePath] => /home/jon/local/lava/htdocs/sites/all/civicrm/extensions/nz.co.fuzion.extendedreport/extendedreport.php
                        )

                    [3] => Array
                        (
                            [prefix] => zombiefinder
                            [filePath] => /home/jon/local/lava/htdocs/sites/all/civicrm/extensions/com.aghstrategies.zombiefinder/zombiefinder.php
                        )

                    [4] => Array
                        (
                            [prefix] => stripe
                            [filePath] => /home/jon/local/lava/htdocs/sites/all/civicrm/extensions/com.drastikbydesign.stripe/stripe.php
                        )

                    [5] => Array
                        (
                            [prefix] => cividiscount
                            [filePath] => /home/jon/local/lava/htdocs/sites/all/civicrm/extensions/org.civicrm.module.cividiscount/cividiscount.php
                        )

                    [6] => Array
                        (
                            [prefix] => shoreditch
                            [filePath] => /home/jon/local/lava/htdocs/sites/all/civicrm/extensions/org.civicrm.shoreditch/shoreditch.php
                        )

                    [7] => Array
                        (
                            [prefix] => flexmailer
                            [filePath] => /home/jon/local/lava/htdocs/sites/all/civicrm/extensions/org.civicrm.flexmailer/flexmailer.php
                        )

                    [8] => Array
                        (
                            [prefix] => mosaico
                            [filePath] => /home/jon/local/lava/htdocs/sites/all/civicrm/extensions/uk.co.vedaconsulting.mosaico/mosaico.php
                        )

                    [9] => Array
                        (
                            [prefix] => noverwrite
                            [filePath] => /home/jon/local/lava/htdocs/sites/all/civicrm/extensions/eu.tttp.noverwrite/noverwrite.php
                        )

                    [10] => Array
                        (
                            [prefix] => sumfields
                            [filePath] => /home/jon/local/lava/htdocs/sites/all/civicrm/extensions/net.ourpowerbase.sumfields/sumfields.php
                        )     
                                   
                    [11] => Array
                        (                                 
                            [prefix] => genderselfidentify                                                                                              
                            [filePath] => /home/jon/local/lava/htdocs/sites/all/civicrm/extensions/org.woolman.genderselfidentify/genderselfidentify.php
                        )                                          
                                 
                    [12] => Array
                        (                         
                            [prefix] => monitoring                                                                                            
                            [filePath] => /home/jon/local/lava/htdocs/sites/all/civicrm/extensions/com.megaphonetech.monitoring/monitoring.php
                        )                                      
                                                                       
                )                                                              
                                                                                       
        )                                                                                                          
                                                                                   
    [expiresCache:protected] => Array                                                    
        (                                                                      
            [civiroot] => 1544739837                                                   
            [default] => 1544739837                                                  
            [full] => 1544739837                                       
            [mapper_moduleFiles] => 1544739837                                     
        )        
                                      
    [table:protected] => civicrm_cache
)
@totten
Copy link
Member

totten commented Dec 13, 2018

Just to cover our bases... the cache is there to avoid filesystem scans. If you manually update the filesystem, then it won't pick up on the new things... until the cache is cleared. Some interesting edge-bits:

  • For cv ext:list, you can pass the refresh option (e.g. cv ext:list -Lr). This should always give a fresh/current view.
  • For cv en, it's clever. First, it checks the cache. If the requested module isn't there, then it clears the cache and checks again. This makes the command more forgiving (s.t. you might not think about the caching much).
  • cv flush should clear it too, although that's heavier than cv ext:list -Lr
  • It would be really tempting for all UI's which list extensions to show a timestamp in the footer (i.e. "Last scanned at XXXX-XX-XX XX:XX")

In your example, could it just be that the Drupal site had a more up-to-date cache than the WP site?

Aside: Operating under the theory that cv ext:list would not show items from the default container on a WP site, I did this little experiment -- download something (eu.tttp.noverwrite) and see if it shows up. Note that it does show up:

[nix-shell:~/bknix/build/wpmaster/wp-content/plugins/civicrm]$ cv ext:list -L
+----------+------------------------------------+-----------------+--------------+-------------+----------------------------------------+
| location | key                                | name            | version      | status      | downloadUrl                            |
+----------+------------------------------------+-----------------+--------------+-------------+----------------------------------------+
| local    | org.civicrm.angularex              | angularex       | 1.0          | uninstalled |                                        |
| local    | org.civicrm.angularprofiles        | angularprofiles | 4.7.31-1.1.2 | installed   |                                        |
| local    | org.civicrm.demoqueue              | demoqueue       | 1.0          | uninstalled | http://FIXME/org.civicrm.demoqueue.zip |
| local    | org.civicrm.shoreditch             | shoreditch      | 0.1-alpha23  | uninstalled |                                        |
| local    | org.civicrm.volunteer              | volunteer       | 4.7.31-2.3.1 | installed   |                                        |
**snip**
|+----------+------------------------------------+-----------------+--------------+-------------+----------------------------------------+

[nix-shell:~/bknix/build/wpmaster/wp-content/plugins/civicrm]$ cv dl noverwrite
Using extension feed "https://civicrm.org/extdir/ver=5.9.alpha1|uf=WordPress|status=stable|ready=ready"
Downloading extension "eu.tttp.noverwrite" (https://github.com/TechToThePeople/noverwrite/archive/1.3.zip)

[nix-shell:~/bknix/build/wpmaster/wp-content/plugins/civicrm]$ cv ext:list -L
+----------+------------------------------------+-----------------+--------------+-------------+----------------------------------------+
| location | key                                | name            | version      | status      | downloadUrl                            |
+----------+------------------------------------+-----------------+--------------+-------------+----------------------------------------+
| local    | eu.tttp.noverwrite                 | noverwrite      | 1.3          | installed   |                                        |
| local    | org.civicrm.angularex              | angularex       | 1.0          | uninstalled |                                        |
| local    | org.civicrm.angularprofiles        | angularprofiles | 4.7.31-1.1.2 | installed   |                                        |
| local    | org.civicrm.demoqueue              | demoqueue       | 1.0          | uninstalled | http://FIXME/org.civicrm.demoqueue.zip |
| local    | org.civicrm.shoreditch             | shoreditch      | 0.1-alpha23  | uninstalled |                                        |
| local    | org.civicrm.volunteer              | volunteer       | 4.7.31-2.3.1 | installed   |                                        |
*snip**
+----------+------------------------------------+-----------------+--------------+-------------+----------------------------------------+

[nix-shell:~/bknix/build/wpmaster/wp-content/plugins/civicrm]$ ls ~/bknix/build/wpmaster/wp-content/uploads/civicrm/ext/

eu.tttp.noverwrite/

@MegaphoneJon
Copy link
Contributor Author

Hmm, this isn't what I'm seeing. I wonder what the difference is. Note that the extensions in the extensions folder show as "Missing":

zabuntu: ~/local/9to5/htdocs(branch:master) » cv ext:list -Lr                                                                                                                                          jon@zabuntu
Refreshing extensions                     
+----------+--------------------------+------+---------+-------------+-------------+
| location | key                      | name | version | status      | downloadUrl |
+----------+--------------------------+------+---------+-------------+-------------+
| local    | com.iatspayments.civicrm | iats | 1.6.2   | uninstalled |             |
| local    | org.civicrm.api4         | api4 | 4.1.0   | uninstalled |             |
+----------+--------------------------+------+---------+-------------+-------------+
------------------------------------------------------------
zabuntu: ~/local/9to5/htdocs(branch:master) » cv api extension.get                                                                                                                                     jon@zabuntu
{                             
    "is_error": 0,            
    "version": 3,          
    "count": 10,       
    "values": [                           
        {                                      
            "key": "org.civicrm.api4",
            "type": "module",
            "name": "API v4",
            "label": "API v4",                  
            "file": "api4",  
            "classloader": [        
                {                    
                    "type": "psr4",
                    "prefix": "Civi\\",
                    "path": "Civi"
                }      
            ],                            
            "requires": [],                    
            "description": "Makes version 4 of the CiviCRM API available to other extensions.",
            "license": "AGPL-3.0",
            "maintainer": {
                "author": "Coleman Watts",        
                "email": "[email protected]"
            },                              
            "urls": {                        
                "Documentation": "https://wiki.civicrm.org/confluence/display/CRM/API+v4+Spec",
                "Licensing": "http://www.gnu.org/licenses/agpl-3.0.html",
                "Main Extension Page": "https://github.com/civicrm/api4"
            },         
            "releaseDate": "2018-10-03",  
            "version": "4.1.0",                
            "develStage": "stable",
            "compatibility": {
                "ver": "5.0"
            },                                  
            "comments": "This extension does nothing on its own. Install it if other extensions require you to do so.",
            "civix": {       
                "namespace": "CRM/Api4"
            },               
            "path": "/home/jon/local/9to5/htdocs/wp-content/plugins/civicrm/civicrm/ext/api4",
            "status": "uninstalled",
            "statusLabel": "",
            "id": "0"                     
        },                                     
        {            
            "key": "com.iatspayments.civicrm",
            "type": "module",
            "name": "iATS Payments",              
            "label": "iATS Payments",
            "file": "iats",                
            "classloader": [],              
            "requires": [],      
            "description": "iATS Payments Web Services Payment Processor",
            "urls": {      
                "Documentation": "https://github.com/iATSPayments/com.iatspayments.civicrm",
                "Getting Started": "https://www.semper-it.com/civicamp-iats-payments-slides",
                "Installation Instructions": "https://github.com/iATSPayments/com.iatspayments.civicrm",
                "Release Notes": "https://github.com/iATSPayments/com.iatspayments.civicrm/blob/master/release-notes/1.6.2.md",
                "Support": "https://github.com/iATSPayments/com.iatspayments.civicrm/issues?state=open"
            },
            "license": "AGPL-3.0",              
            "maintainer": {  
                "author": "Alan Dixon",
                "email": "[email protected]"
            },                  
            "releaseDate": "2018-08-15",
            "version": "1.6.2",
            "develStage": "stable",
            "compatibility": {            
                "ver": "5.0"                   
            },       
            "comments": "This release resolves some 5.x compatibility issues, but should continue to work on 4.7.x. Please see the current release notes linked above for details.\n",
            "civix": {
                "namespace": "CRM/iATS"
            },                                              
            "path": "/home/jon/local/9to5/htdocs/wp-content/plugins/civicrm/civicrm/ext/iatspayments",
            "status": "uninstalled",            
            "statusLabel": "",
            "id": "1"
        },
        {
            "key": "org.civicrm.shoreditch",
            "type": "module",
            "name": "CiviCRM Bootstrap theme",
            "label": "CiviCRM Bootstrap theme",
            "file": "shoreditch",
            "classloader": [],
            "requires": [],
            "path": "",
            "status": "installed-missing",
            "statusLabel": "Enabled (Missing)",
            "id": "2"
        },
        {
            "key": "org.civicrm.flexmailer",
            "type": "module",
            "name": "FlexMailer",
            "label": "FlexMailer",
            "file": "flexmailer",
            "classloader": [],
            "requires": [],
            "path": "",
            "status": "installed-missing",
            "statusLabel": "Enabled (Missing)",
            "id": "3"
        },
        {
            "key": "uk.co.vedaconsulting.mosaico",
            "type": "module",
            "name": "Mosaico",
            "label": "Mosaico",
            "file": "mosaico",
            "classloader": [],
            "requires": [],
            "path": "",
            "status": "installed-missing",
            "statusLabel": "Enabled (Missing)",
            "id": "4"
        },
        {
            "key": "coop.palantetech.electoral",
            "type": "module",
            "name": "Electoral API",
            "label": "Electoral API",
            "file": "electoral",
            "classloader": [],
            "requires": [],
            "path": "",
            "status": "installed-missing",
            "statusLabel": "Enabled (Missing)",
            "id": "5"
        },
        {
            "key": "com.cividesk.email.sparkpost",
            "type": "module",
            "name": "SparkPost integration",
            "label": "SparkPost integration",
            "file": "sparkpost",
            "classloader": [],
            "requires": [],
            "path": "",
            "status": "installed-missing",
            "statusLabel": "Enabled (Missing)",
            "id": "6"
        },
        {
            "key": "com.drastikbydesign.stripe",
            "type": "module",
            "name": "Stripe",
            "label": "Stripe",
            "file": "stripe",
            "classloader": [],
            "requires": [],
            "path": "",
            "status": "installed-missing",
            "statusLabel": "Enabled (Missing)",
            "id": "7"
        },
        {
            "key": "com.megaphonetech.monitoring",
            "type": "module",
            "name": "Megaphone Monitoring",
            "label": "Megaphone Monitoring",
            "file": "monitoring",
            "classloader": [],
            "requires": [],
            "path": "",
            "status": "installed-missing",
            "statusLabel": "Enabled (Missing)",
            "id": "8"
        },
        {
            "key": "net.ourpowerbase.sumfields",
            "type": "module",
            "name": "Summary Fields",
            "label": "Summary Fields",
            "file": "sumfields",
            "classloader": [],
            "requires": [],
            "path": "",
            "status": "installed-missing",
            "statusLabel": "Enabled (Missing)",
            "id": "9"
        }
    ]
}
--------

@totten
Copy link
Member

totten commented Dec 13, 2018

If they're all missing, then maybe something went wonky with the parent dir's config? Might be worth double-checking the paths:

[nix-shell:~/bknix/build/wpmaster/wp-content/plugins/civicrm]$ cv ev 'return [Civi::settings()->get("extensionsDir"), CRM_Core_Config::singleton()->extensionsDir, CRM_Extension_System::singleton()->getDefaultContainer()->baseDir]'
[
    "[civicrm.files]/ext/",
    "/Users/totten/bknix/build/wpmaster/wp-content/uploads/civicrm/ext/",
    "/Users/totten/bknix/build/wpmaster/wp-content/uploads/civicrm/ext"
]

@MegaphoneJon
Copy link
Contributor Author

I think this is the same bug referenced in https://lab.civicrm.org/dev/core/issues/385. I'm not sure I'm going to move on this at the moment, but I'm cross-referencing them in case one day someone does :)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
2 participants
@totten @MegaphoneJon