Parsing Gemfile and gemspec files to extract dependencies and their groups

[nareshdevasani]

Hello friends, need help to extract dependencies and the group each dependency belongs to.

Any ruby project can contain only Gemfile or both Gemfile and gemspec file. I see we can declare the dependencies(gems) in both files. While Gemfile has groups like development/test etc, gemspec has development_dependency vs dependency methods.

Now, I would like to list the dependencies declared in the above to files.

• Are there any bundler/gem commands to list the dependencies without installing them?
• How can I get the similar dependencies shown by the below page using the the given repository from(https://github.com/rgrove/crass.git)?:
  https://rubygems.org/gems/crass/versions/1.0.6/dependencies

Kindly provide pointers.

I tried generating GEM for the repository, but it is not retaining all the dependencies from the gemspec file to the newly created gemspec within the gem.

[mohits]

Does this help?
https://notepad.onghu.com/2020/ruby-tricks-003-get-gem-deps/

[nareshdevasani]

How does below page gets the dependencies at first place:
https://rubygems.org/gems/crass/versions/1.0.6/dependencies

It is part of rubygems.org, then how actually the dependency data was uploaded?

[mohits]

Fairly likely, that it was also created using a Ruby script. What's the problem if a Ruby script is used?

[nareshdevasani]

Reason is our automation is written in golang which interacts with all internal projects including ruby project. How can I transform Gemfile or gemspec file to the gemspec file needed by rubygems.org?

In a way my requirement is below:
How can I transform gemspec having below code:

  spec.add_dependency "dependabot-terraform", Dependabot::VERSION

  common_gemspec.development_dependencies.each do |dep|
    spec.add_development_dependency dep.name, *dep.requirement.as_list
  end

to the gemspec having below:

  s.add_runtime_dependency(%q<dependabot-terraform>.freeze, ["= 0.215.0"])
  s.add_development_dependency(%q<debug>.freeze, [">= 1.0.0"])

[mohits]

If I'm honest, I am struggling to understand what you want to do. Do you want to start with a Gemfile or a Gemfile.lock?
The code you posted is actually just Ruby code and the versions are almost equivalent in Ruby code.

Rubygems does dependency resolution and management. If you choose to not use Ruby, you will need to rewrite the parser in Golang for your needs. You can try to follow the code that I linked to - but I expect that you will need to translate all the pieces out to Golang for it to actually work.

On the other hand, if you have a Gemfile.lock, you can read up the syntax and it might be much easier to parse.

I fear that I am unable to provide any more help but do post back if you find something useful.

[nareshdevasani]

Yes I agree that parsing Gemfile.lock is easier than parsing Gemfile or gemspec file(with ruby code).

One reason why I need to parse Gemfile.lock/gemspec files: I cannot identify the development dependencies from the lock file.
Where as Gemfile has groups named development or test. So I can easily understand that these are development dependencies.
Also .gemspec file has add_dependency or add_development_dependency, so based on that we could identify dev dependencies.

So now what I am looking at is simple:

Given my ruby project with Gemfile/.gemspec and Gemspec.lock files, how can I easily extract dependencies and their details like below:

• Dev dependencies
• Production dependencies
• Versions of dependencies

Are there any tools/CLI provided by rubygems/bundler/gem?

Thanks again for your help.

[simi]

Is there any problem using DB snapshot I pointed you to before (if I remember well)? You'll not be limited for your requests than.

https://rubygems.org/gems/crass/versions/1.0.6/dependencies

This page is based on the DB data publicly available using this code to render https://github.com/rubygems/rubygems.org/blob/master/app/controllers/dependencies_controller.rb + https://github.com/rubygems/rubygems.org/blob/master/app/views/dependencies/show.html.erb.

[nareshdevasani]

Hi Simi, the DB dump is only for the GEMS available on the rubygems.org. And thanks for the reference to DB dump.

My requirement is to parse my own project's dependencies which is not a GEM uploaded in rubygems.org.

So I am curious to know how in general the dependencies are uploaded to rubygems so that, I might get an answer to parse my own project's dependencies.