All notable changes to this project will be documented in this file. This project adheres to Semantic Versioning and Keep a Changelog.
- !1664 Automatically populate
env
properties on newly added custom pluginsettings
inmeltano.yml
- !1664 Have
meltano config <plugin> list
print default value along with setting name and env var - !1664 Pass configuration environment variables when invoking plugins
- !1664 Set
MELTANO_EXTRACTOR_NAME
,MELTANO_EXTRACTOR_NAMESPACE
, andMELTANO_EXTRACT_{SETTING...}
environment variables when invoking loader or transformer - !1664 Set
MELTANO_LOADER_NAME
,MELTANO_LOADER_NAMESPACE
, andMELTANO_LOAD_{SETTING...}
environment variables when invoking transformer - !1664 Allow dbt project dir, profiles dir, target, source schema, target schema, and models to be configured like any other plugin, with defaults based on pipeline-specific environment variables
- #2029 Allow target-postgres and target-snowflake schema to be overridden through config, with default based on pipeline's extractor's namespace
- !1664 Infer compatibility between extractor and transform based on namespace rather than name
- !1664 Determine transform dbt model name based on namespace instead of than replacing
-
with_
in name - !1664 Don't pass environment variables with "None" values to plugins if variables were unset
- !1664 Determine Meltano Analyze schema based on transformer's
target_schema
or loader'sschema
instead ofMELTANO_ANALYZE_SCHEMA
env var
- #2028 Improve descriptions and default values of
meltano add --custom
prompts
- #2042 Fix bug causing Connection Setup UI to fail when plugin docs URL is not set
- #2045 Hide plugin logo in UI if image file could not be found
- #2043 Use plugin name in UI when label is not set, instead of not showing anything
- #2044 Don't show button to open Log Modal on Pipelines page if pipeline has never run
- #2024 Have plugin venvs not inherit Meltano venv to prevent wrong versions of modules from being loaded
- #2019 Ask for setting names when adding a new custom plugin
- #2011 Make tap-gitlab private_token setting optional for easier extraction of public groups and projects
- #2012 Add target-jsonl loader
- #2010 Fix bug causing dot-separated config keys to not be nested in generated tap or target config
- #2020 Fix bug that caused
meltano select
to addselect
option to every plugin inmeltano.yml
instead of just the specified one - #2021 Only ask for capabilities when adding a custom extractor, not a loader or other plugin
- #1987 Restore GitLab and Zendesk data sources in UI
- #2005 Add "Don't see your data source here?" option in UI
- #2008 Clarify that pipelines UI only supports target-postgres
- #2007 Don't install airflow, dbt and target-postgres by default as part of 'meltano init'
- #2007 Only run 'airflow scheduler' as part of 'meltano ui' when airflow is installed
- #2007 Install airflow, dbt, and target-postgres on DigitalOcean images
- #1985 Fixed bug with Airflow installs WTForms 2.3.0 instead of 2.2.1, which is incompatible
- #1953 Show design attribute descriptions in tooltips in report builder
- #1787 Show Shopify extractor in UI
- !1611 Only show design description if it is different from design label
- #1948 Show Intercom button in bottom right on MeltanoData.com instances
- #1930 Add button to remove report from dashboard when editing dashboard
- #1845 Add button to delete report to report builder interface
- #1849 Add button to rename report to report builder interface
- #1951 Add button to edit dashboard name and description to dashboard page
- !1607 Move date range picker into results area of report builder interface
- !1608 Make report title more prominent in report builder
- #1940 Show Google Ads extractor in UI
- #1941 Suggest disabling ad blocker if inline docs iframe for an Ads or Analytics extractor failed to load
- #1667 Have 'meltano init' create system database and install airflow, dbt, and target-postgres plugins
- #1942 Ensure navigation bar is hidden in production when docs are viewed inline
- #1937 Add optional
plugin_type
argument tomeltano install
to only (re)install plugins of a certain type
- #1938 Display error message when viewing dashboard before pipeline has run
- #1938 Fix regression causing dashboards and reports not to load when readonly mode is enabled (like on the demo instance)
- #1936 Fix regression causing UI to fail when analytics/tracking is enabled
- #1477 Allow read-only mode and authentication to be used at the same time, to allow anonymous read-only access and only require authentication for write actions.
- #1914 Allow default dashboards and reports to be updated in place if package contains snapshots of older versions
- #1933 Allow Meltano UI Google Analytics ID to be overridden using environment variable
- #1896 Set pipeline update interval to daily by default, to start after first successful manual run
- #1888 Explain in "Edit Connection" button tooltip why it may be disabled
- #1890 Clarify that changing Start Date requires a new pipeline to be set up
- #1892 Clarify in Run Log modal that the "Explore" button can also be found on the Connections page
- #1891 Show data source logo and label in Run Log modal header insteadof pipeline ID
- #1893 Hide "Download Log" button while pipeline is running instead of disabling it
- #1894 Suggest connecting a data source on Pipelines page when there are no pipelines yet
- #1912 Suggest user gets in touch if the report they're looking for is not included by default
- #1911 Display "Last updated: Updating..." instead of "Last updated: 1969-12-31" on reports while pipeline is running
- #1910 Fix pipeline "Start date" and report "Data starting from" off-by-1 errors caused by timezone differences
- #1909 Suggest disabling ad blocker if request related to an Ads or Analytics extractor was blocked by browser
- #1886 Don't prepopulate date fields that are not required and are better left blank
- #1887 Hide End Date fields in connection setup since our end-users will want to import everything
- #1905 Hide Google Analytics Reports field from UI since startup founder end-users will stick with default
- #1920 Fix extractor logo on Google Analytics Explore page
- #1895 Fix bug causing newly created pipeline not to show as running when it is
- #1906 Fix "Test Connection" for extractors that require a file to be uploaded, like Google Analytics
- #1931 Validate uploaded file path when saving or testing connection settings
- #1883 Fix dashboard and embedded reports failing to load when design has no joins
- #1854 Remove non-marketing-sales data sources from UI
- #1881 Store in system database when user was last active
- #1846 Freeze reports with relative date filters in time when shared or embedded
- #1847 Show date range on embedded reports and dashboards
- #1847 Show date range on reports on dashboards
- #1188 Allow sorting by timeframe period columns (e.g. "Creation Date: Month", "Creation Date: Year")
- #1873 Display error message when viewing model/design/report before pipeline has run
- #1874 Print full error when initial model compilation fails
- #1875 Automatically run query when sorting is changed
- #1876 Don't store Analyze UI state in report file
- #1877 Allow designs to reference the same table more than once
- #1878 Recompile models when meltano is upgraded
- #1799 Improve date range UX by displaying the date range associated with each attribute in the
<select>
(previously the user had to manually check each one-by-one to see if it had an associated date range filter)
- #1799 Update "Date Range(s)" button label to account for pluralization
- #1799 Fallback to inline text and only display the date range
<select>
if there are two or more date ranges to filter on - #1799 Update date range picker to initialize at the first attribute with a valid date range
- #1799 Update the Report Builder's "jump to date range dropdown" buttons (small calendar icon button associated with the left pane's attribute items) to automatically focus the date range that's associated
- #1872 Delete state left over from different pipeline run for same extractor
- #1779 Fix loading report directly by URL when design request completes before reports request
- #1843 Update the Google Ads Extractor selected attributes definition to also extract the Ad Network and Device segments for the Ads Performance Reports.
- #1852 Move Pipelines after Connections in navbar
- #1850 Rename Connections tab "Connection" and "Pipeline" buttons to "Edit Connection", and "View Pipeline"
- #1856 Remove "Custom" data source option from UI
- #1867 Make timeframe table headings more human-friendly
- #1848 Fix Explore page "Report Builder" column loading when model name and model topic name do not match
- !1523 Add support for relative date filter definitions to Meltano Filters. That means that filters over dates and times can have a
[+-]N[dmy]
format instead of a fixed date. That allows Meltano to generate a date relative to a pivot date provided by in the query definition orNOW()
. - #1830 Add relative vs. absolute date ranges to date range picker of Report Builder
- #1830 Update date ranges calendars with "Today" marker for improved UX
- #1831 Change main navigation "Reports" to "Explore" and update its nested CTAs to link to a landing page per data source
- #1705 Remove
meltano permissions
feature now that it has been extracted into https://gitlab.com/gitlab-data/permifrost. - Updated "Report Builder" page with a header to better communicate what the page is for.
- #1840 Format InputDateIso8601 value as YYYY-MM-DD since a full timestamp value could represent a different date in UTC and local timezone
- #1842 Fix empty filter attributes bug for non-join designs
- #1820 Add Vertical Bar chart type to Report chart options
- #1820 Updated chart type selection as a dropdown for improved UX (ensures the chart icon is adorned with its label)
- #1837 Fix tap-mongodb database name setting
- #1838 Properly handle dates and timezones in date range picker
- #1838 Ensure records on boundary dates are included when using date range picker with column of type "time"
- #1836 Don't crash when gunicorn is sent HUP signal to reload Meltano service
- #1601 Add Explore landing pages per data source to act as an aggregate jump-off point to related dashboards, reports, report templates, and more
- #1601 Change "Reports" CTA in each Pipeline and the JobLog modal to link to its corresponding and newly added Explore page
- #1698 Change information architecture to separate Connections and Pipelines into distinct pages
- #1811 Fix an issue when installing a custom plugin.
- #1794 Remove the notification field when notifications are disabled.
- #1815 Fix
mapActions
misplacement incomputed
vs.methods
- #1468 Update asn1crypto to get Meltano to work on macOS Catalina
- #1809 Fix LogModal padding render issue and
TypeError
with proper conditional check prior to dereferencing - #1810 Fix an issue where Notifications would not be sent when the application used multiple workers
- #1783 Add Shopify extractor as a hidden plugin
- #1499 Add date range selector to Analyze UI (requires a
type=date
ortype=time
in each model needing this functionality)
- #1777 Update Meltano Analyze to only preselect the first column and aggregate attributes when no attributes have a
require
d setting - #1796 Update date range and filter changes to trigger autorun if enabled
- #1798 Add OK button to toasts that couldn't be dismissed previously, to prevent them from getting in the way of modal buttons
- #1803 Ensure SMTP credentials can be set via environment variables
- #1778 Fix missing pipeline date when visiting page directly from URL
- #1646 Add default Stripe dashboard
- #1759 Add default reports and dashboard for Google Ads data
- #1775 Add default dashboard for GitLab extractor
- #1714 Add support for a
required
setting in Models so Analyze can still work with more complex reporting scenarios (Facebook and Google Adwords need this) - #1780 Add default reports and dashboard for Facebook Ads data
- #1740 Add "Sharing Reports and Dashboards" section to Getting Started guide
- #1484 Add a subscription field to be notified when a Pipeline will be completed.
- #1740 Update Getting Started guide screenshots with up-to-date UI
- #1751 Custom report ordering now works based on user customization
- #1756 Fix embed app to properly render based on
report
ordashboard
type
- #1754 Fix duplicate "Share" button and Reports dropdown clipping issue
- #609 Add the Google Ads Extractor to Meltano as a hidden plugin. It will be fully enabled on Meltano UI once OAuth support is added. It uses the tap defined in https://gitlab.com/meltano/tap-adwords/
- #1693 Add default transformations for the Google Ads Extractor. They are using the dbt package defined in https://gitlab.com/meltano/dbt-tap-adwords
- #1694 Add default Meltano Models for the Google Ads Extractor. They are defined in https://gitlab.com/meltano/model-adwords
- #1695 Add documentation for the Google Ads Extractor
- #1723 Add various mobile and widescreen related style tweaks to improve base layout at mobile and widescreen widths
- !1460 Remove the FTP access from Meltano hosted instances
- #1629 Add "Share Dashboard" functionality
- #1629 Update report "Embed" button to "Share" and include a share link to accompany the embed snippet
- #1680 Fix initial "Last Run" button of a pipeline run to properly open the corresponding job log
- #1650 create TOS page and add TOS link to website footer
- #1681 Update
transform
during pipeline save to conditionally setskip
vs.run
to prevent wasted cycles for extractors that lack transformations - #1696 Update dashboards list to be alphabetically sorted
- #1710 Hide
tap-fastly
in UI
- #1696 Fix duplicate chart renders when dashboard is loaded
- #1696 Fix "Add to Dashboards" button when loading an existing report (additionally updated
disabled
button states) - #1711 Disable fields of all kinds when a plugin setting is protected or set in env or meltano.yml
- #1712 Fix lock icon tooltip message on plugin settings that were set in env or meltano.yml
- #1677 Properly represent values of boolean settings that were set using environment verariables in UI
- #1682 Use human-readable update interval labels
- #1514 Remove DBT docs integration
- #1679 Prevent the
hidden
settings from being sent to the front-end, potentially causing configuration failure
- #1675 Fix future grant diffing for databases and schemas
- #1674 Fix duplicate pipelines bug resulting from recent addition to view and update existing connections
- #1672 Pin Werkzeug version to 0.16.1 since 1.0.0 is unsupported by Flask-BabelEx
- #1671 Fix error handling bug that caused a console error that impacted further UI interaction
- #1545 Add read-only report embed functionality via embeddable
iframe
copy-to-clipboard snippet - #1606 Update UI after successful plugin configuration with auto installed reports and dashboards
- #1614 Add 'Fix Connection' and 'View Connection' CTAs to Integrations with corresponding pipelines
- #1550 Add the Meltano OAuth Service integration to manage the OAuth flow in the plugin configuration
- #1594 Improve onboarding UX by moving the "Update Interval" selection to a post-successful-pipeline action
- #1594 Update pipelines to be sorted alphabetically to match data sources organization
- #1659 Update query attribute toggling and results UX when autorun query is on (via 500ms debounce)
- #1475 GitLab extractor in the UI steers user towards a single data source
- #1657 Fix
update_dashboard
error when payload lacked anew_settings
key - #1602 Fix instances where
<a disabled='...'>
vs.<button disabled='...'>
didn't functionally disable the button (previously they were only disabled visually) - #1656 Fix conditional header in docs to support Meltano.com and inline docs within the Meltano app
- #1154 Adds non-dry mode to
meltano permissions
on Snowflake so that queries can be executed - #1578 User can request help to delete their data from their MeltanoData instance
- #1516 Pipelines now show extractor label rather than name
- #1652 Removes the
--full-refresh
command frommeltano permissions
- #1595 Updates
meltano permissions
to only revoke permissions on databases defined in the spec - #1588 Update
scrollTo
behavior in Job Log to work across browsers - #1660 Fix minor action/mutation bug when loading a report in Analyze
- #1607 Fix inaccurate error during report additions/removal from dashboards (via refactor SSOT reports store)
- #1625 Update docs on meltano.com to only include extractors and loaders provided in the hosted version of Meltano.
- #1590 Add additional targets to
dbt clean
- #1655 Add UX message to close buttons in Job Log Modal to reinforce that the pipeline still runs after closing (Ben's hover idea)
- #1618 Fix an issue where an expired session would not redirect to the Login page
- #1630 Fix an integrations setup bug that prevented subsequent pipelines to be created unless a full page refresh occurred
-
#1462 User will be able to reorder dashboard reports
-
#1482 Add future grants and revocations for schemas, tables, and views for roles in the
meltano permissions
command -
#1376 Add last updated date to reports
-
#1409 Add data start date to Analysis page
-
#1241 Add
dashboard
plugin type to enable bundling curated reports and dashboards for data sources -
#1241 Add
--include-related
flag tomeltano add
andmeltano install
to automatically install related plugins based on namespace -
#1241 Add default dashboard and reports for Google Analytics
- #1481 Add table and view revocations for roles in the
meltano permissions
command - #1459 Users can no longer install tap-carbon-intensity from the UI
- #1600 Fix tooltip for Data Source "Connect" buttons
- #1605 Fix an infinite loop causing extraneous API calls to the configuration endpoint
- #1561 Fix
onFocusInput()
to properly focus-and-auto-scroll to<input type='file'>
s in the data source docs UI - #1561 Fix
<input type='file'>
styling to better accommodate flexible widths
- #1592 Add MAX and MIN aggregate functions to Meltano Models
- #1552 Add "Custom" data source CTA to link to the create custom data source docs
- #1462 User will be able to reorder dashboard reports
- #1510 Remove breadcrumbs (not currently useful)
- #1589 Add dbt-specific files to a .gitignore
- #1402 Onboarding redesign to minimize steps and friction ('Extractors' as 'Data Sources', pipelines are secondary to 'Data Source' integrations, and removed loader, transform, and pipeline name as editable in favor of preselected values in accordance with our hosted solution)
- #1402 Local development now requires
.env
to connect atarget-postgres
loader (docs update to follow in #1586 ) - #1410 Update the Design UI to expose timeframes explicitly
- #1573 Fix docs
shouldShowNavbar
conditional and improve query stringembed=true
parsing - #1579 Make color contrast for CTA buttons accessible
- #1410 Fix a problem with Report that has timeframes selections
- #1556 Add default transformations for the Facebook Ads Extractor. They are using the dbt package defined in https://gitlab.com/meltano/dbt-tap-facebook
- #1557 Add default Meltano Models for the Facebook Ads Extractor. They are defined in https://gitlab.com/meltano/model-facebook
- #1560 Make the Facebook Ads Extractor available by default on Meltano UI
- #1541 Revert
tap-csv
'skind: file
to text input forcsv_files_definition
as we don't fully supporttap-csv
via the UI with single (definition json) and multiple (csv files) file uploading - #1477 Add a
read-only
mode to Meltano to disable all modifications from the UI
- #608 Add the Facebook Ads Extractor to Meltano as a hidden plugin. It will be fully enabled on Meltano UI once bundled Transformations and Models are added. It uses the tap defined in https://gitlab.com/meltano/tap-facebook/
- meltano/model-stripe#2 Add timeframes to the Stripe models
- #1533 Add documentation for the Facebook Ads Extractor
- #1527 Update the dashboard modal header to properly differentiate between "Create" and "Edit"
- #1456 404 Error page now has better back functionality and ability to file new issues directly from the page
- #1538 Fix timeframes not properly displaying on the base table
- #1574 Fix an issue with Meltano crashing after a succesful login
- #1568 Restore support for custom plugins that don't have their available settings defined in discovery.yml
- #1483 Add login audit columns to track last login time
- #1480 Add tests to
meltano permissions
command for Snowflake - #1392 Add inline docs to Extractor configurations in iteration toward improving data setup onboarding
- #1480 Add schema revocations for roles in the
meltano permissions
command - #1458 Remove tap-carbon-intensity-sqlite model from default installation
- #1458 Update docs to reflect new getting started path and updated screenshots
- #1513 Remove dead code related to
/model
route that we no longer link to in favor of the contextual Analyze CTAs and theMainNav.vue
's Analyze dropdown - #1542 Update version, logout, and help UI partial (upper right) to have less prominence and more clearly communicate the "Sign Out" action
- #1480 Fix database revocations corner case for roles in the
meltano permissions
command - #1553 Fix bug occurring when loading a report that lacks join tables
- #1540 Meltano Analyze will now leverage Pipelines instead of Loaders in the connection dropdown
- #1540 Meltano Analyze will now infer the connection to use instead of it being provided by the user
- #1521 Sanitize user-submitted string before using it in file path
- #1391 Lock all settings that are controlled through environment variables
- #1393 Add contextual Analyze CTAs for each Pipeline in the Pipelines list
- #1551 Add dbt clean before compile and runs
- #1424 Update pipeline elapsed time display to be more human friendly
- #1430 Fix the state not stored for pipelines when Transforms run
- #1448 Fix
AnalyzeList.vue
to display message and link when lacking contextual models
- #1520 Fix bug when updating a dashboard that could undesirably overwrite another existing dashboard
- #1461 Display toasted notification for report adding to dashboard
- #1419 Add ability to edit and delete dashboards
- #1411 Add download log button to Job Log Modal
- #1311 Remove unused meltano/meltano/runner docker image
- #1502 Update configuration file uploads to occur on save vs. file picker completion
- #1518 Fix bug that caused all text fields to show up as required in configuration modals
- #1446 Fix bug that could result in a broken report when the report URL was manually modified
- #1411 Fix bug when reading too large a job log file
- #1269 Add
kind: file
so single file uploads can be used with extractors (tap-google-analytics
'skey_file_location
is the first user) - #1494 Add
LIKE
options to Analyze Filter UI so users better understand what filtering patterns are available
- #1399 Log Modal now has a prompt to explain potential factors in required time for pipelines to complete
- #1433 Remove
/orchestrate
route and thus the Airflow iframe as this is overkill for our current target users
- #1434 Fix Analyze CTAs to only enable if at least one related pipeline has succeeded
- #1447 Various fixes around loading and reloading reports to mitigate false positive
sqlErrorMessage
conditions - #1509 Allow plugin profile config to be set through meltano.yml
- #1437 Users can now share their dashboards with an automatically generated email
-
#1466 Filters now have clear language and indiciation that they use AND for chaining
-
#1464 Remove the "only" option for transforms in Create Pipeline form
-
#1399 Log Modal now has a prompt to explain potential factors in required time for pipelines to complete
-
#1431 Add "pipeline will still run if modal is closed" message in the Job Log Modal
- #1422 Update start date field to have a recommendation
- #1447 Various fixes around loading and reloading reports to mitigate false positive
sqlErrorMessage
conditions - #1443 Fix tooltip clipping in modals
- #1500 Fix
meltano install
not running the migrations.
- #1403 Remove "Orchestrate", "Model", and "Notebook" from the main navigation until each respective UI is more useful (the
/orchestrate
and/model
routes still exist) - #1476 Add database and warehouse revocations for roles in the
meltano permissions
command - #1473 Update Release issue template to recent guidelines
- #1374 Add role revocation for users and roles in the
meltano permissions
command - #1377 Document cleanup steps after MeltanoData testing
- #1438 Add documentation for DNS spoofing error
- #1436 Add video walkthrough on how to setup Google Analytics so that the Meltano Extractor can be able to access the Google APIs and the Google Analytics data.
- #1350 Switch to all lower case for Snowflake permission comparisons in the
meltano permissions
command - #1449 Hide the Marketo Extractor form Meltano UI
- #1397 Optimize workflow for MeltanoData setup
- #1423 Update sidebar and docs to include Ansible
- #1435 Change "Model" to "Analyze" so the Pipeline CTA is actionable and less abstract
- #1432 Changed "Close" to "Back" in Log Modal to help mitigate "Am I ending the pipeline?" concerns
- #1439 Fix relative elapsed time since last run time display in the Pipelines UI
- #1441 Fix auto advance to "Create Pipeline" when coming from "Load" step (previously "Transform" step, but this has been removed from the UI)
- #1440 Allow installed plugins to appear in UI even if hidden in configuration
- #1351 Add "Create Meltano Account" promo for
meltano.meltanodata.com
- #1055 Add "Disable" button to Tracking Acknowledgment toast so user's can opt-out from the UI
- #1408 Add "Last Run" context to each pipeline
- #1408 Add "Started At", "Ended At", and "Elapsed" to Job Log modal
- #1390 Display of extractors and loaders can now be configured through the
hidden
property indiscovery.yml
- #1398 Update default Transform from "Skip" to "Run"
- #1406 Update Analyze Query section CSS for improved UX (visually improved organization and scanability)
- #1417 Update SCSS variable usage in components for SSOT styling
- #1408 Updated date and time displays to be human-friendly (
moment.js
) - #1268 Remove Transform step from UI (Create Schedule still allows choosing "Skip" or "Only" but will intelligently default to "Skip" or "Run")
- #1361 Add
kind: hidden
todiscovery.yml
so certain connector settings can validate with a defaultvalue
but remain hidden from the user for improved UX
- #1389 Temporary Profiles feature removal (conditionally removed if 2+ profiles not already created so existing users can continue using multiple profiles if created)
- #1373 Update MeltanoData deletion process with 1Password
- #1401 Fix double instance of self hosted CTA on desktop sites
- #1371 Provide more specific instructions for Google Analytics configuration
- #1381 Update the default directory for client_secrets.json for the Google Analytics Extractor to be located under the extract/ directory and not the project's root.
- #1345 Update the documentation for the Salesforce Extractor to contain additional information on Security Tokens
- #1383 Add CTA for hosted solution signup to navigation
- #1379 Fix an issue with Airflow scheduling too many jobs.
- #1386 Fix connector modal clipping issue where small browser heights prevented accessing the "Save" area
- #1373 Update MeltanoData deletion process with 1Password
- #1373 Update Analyze dropdown as scrollable to better display model CTAs (scrollable dropdown vs. scrolling entire page)
- #1373 Fix formatting on custom containers in MeltanoData guide
- #1343 Add current Meltano version to main navigation
- #1358 Update MeltanoData guide with maintenance and debugging instructions
- #1337 Add CTA to installations for free hosted dashboards
- #1365 Add process for deleting meltanodata instances
- #1340 Update connector settings UI to communicate the required status of each setting
- #1357 Update LogModal Analyze CTAs so Analyze can preselect the correct loader for a given analysis
- #1364 Fix instructions to SSH into MeltanoData.com instance
- #1355 Upgrade version of
discovery.yml
so that not upgraded Meltano instances with a pre v1.9.0 Meltano version do not break.
- marketing#103 Add Google Site Verification token to site
- #1346 Add new tutorial for using FileZilla with a Meltano project
- #1292 Add guide for setting up Meltano projects on meltanodata.com
- #1341 Various
discovery.yml
and connector configuration UI updates to improve UX. - #1341 Updated documentation to communicate the various optional settings of a connector
- #1334 Fix automatic population of airflow.cfg after installation
- #1344 Fix an ELT automatic discovery error when running Meltano on Python3.6
- #764 Add plugin profiles to enable multiple configurations for extractors
- #1081 Add ability to delete data pipelines
- #1217 Add "Test Connection" button to validate connection settings prior to ELT runs
- #1236 Add contextual Analyze CTAs in the Job Log UI
- #1271 Add labels in discovery.yml for easy brand definition
-
#1323 Add CTA to send users to Typeform to provide info for setting up a hosted dashboard
-
#1323 Add CTA to send users to Typeform to provide info for setting up a hosted dashboard
-
#1271 Improve messaging on tap and target settings modals
-
#1226 Update Pipelines main navigation link to show all data pipeline schedules if that step has been reached
-
#1323 Add CTA to send users to Typeform to provide info for setting up a hosted dashboard
-
#1271 Improve messaging on tap and target settings modals
-
#1246 Update the Salesforce API + Postgres Tutorial to use Meltano UI for setting up the Extractor and Loader, running the ELT pipeline and analyzing the results.
-
#1225 Update dbt docs link to be conditional so the user doesn't experience 404s
- #1318 Pin dbt version to
v0.14.4
to address Meltano Transformation failing when using dbtv0.15.0
- #1184 Fix
contextualModels
implementation for contextual CTAs in Job Log modal
- #1236 Add contextual Analyze CTAs in the Job Log UI
- #1298 Let default entity selection be configured in discovery.yml under
select
- #1298 Define default entity selection for tap-salesforce
- #1304 Fix Meltano subprocess fetching large catalogs (e.g. for Salesforce) getting stuck do to the subprocess' stderr buffer filling and the process getting deadlocked.
- #1235 Add help link button in the app
- #1285 Add link to YouTube guidelines for release instructions
- #1277 Move sections that don't apply to outside contributors from Contributing and Roadmap docs to Handbook: Release Process, Release Schedule, Demo Day, Speedruns, DigitalOcean Marketplace
- #1257 Prevent modified logo file upon each build
- #1289 Dismiss all modals when using the escape key
- #1282 Remove Entity Selection from the UI (still available in CLI) and default to "All" entities for a given data source
- #1303 Update the configuration options for the Salesforce Extractor to only include relevant properties. Remove properties like the client_id that were not used for username/password authentication.
- #1308 Update the configuration options for the Marketo Extractor to use a Start Date instead of a Start Time.
- #1297 Get actual latest ELT job log by sorting matches by creation time with nanosecond resolution
- #1297 Fix pipeline failure caused by jobs that require true concurrency being executed on CI runners that don't
- #1222 Include static application security testing (SAST) in the pipeline
- #1164 Add "transform limitations" message to Transform UI
- #1272 Add Vuepress plugin to generate a sitemap on website build
- meltano-marketing#89 Adds basic title and meta descriptions to all public-facing website & documentation pages.
- #1239 Update header buttons layout on small viewports
- #1019 Automatically update package.json file versions
- #1253 Do not allow
meltano
command invocation without any argument - #1192 Improve helper notes associated with each Extract, Load, and Transform step to better communicate the purpose of each
- #1201 Improved "Auto Advance" messaging regarding Entity Selection. We also doubled the default toast time to improve likelihood of reading feedback.
- #1191 update Google Analytics extractor documentation to explain how to set up the Google Analytics API, and remove duplicate instructions from the Google Analytics API + Postgres tutorial
- #1199 Add example and sample CSV files to the CSV extractor documentation
- #1247 Update the Loading CSV Files to a Postgres Database Tutorial to use Meltano UI for setting up the Extractor and Loader, running the ELT pipeline and analyzing the results. Also provide all the files used in the tutorial (transformations, models, etc) as downloadable files.
- [#1279] Revise "Roadmap" section of the docs with clarified persona, mission, vision, and re-order content
- #1134 Update the GitLab API + Postgres. Include video walk-through and update the end to end flow to only use Meltano UI.
- #95 Update the DigitalOcean CTA to go to the public directory page for the Meltano droplet
- #1270 Main navigation "Pipeline" to "Pipelines" to reinforce multiple vs. singular (conflicts a bit with the verb approach of the other navigation items but we think it's worth it for now)
- #1240 Provide clarity around how Airflow can be used directly in documentation and UI
- #1263 Document lack of Windows support and suggest WSL, Docker
- #1259 Fix
meltano elt
not properly logging errors happening in the ELT process - #1183 Fix a race condition causing the
meltano.yml
to be empty in some occurence - #1258 Fix format of custom extractor's capabilities in meltano.yml
- #1215 Fix intercom documentation footer overlap issue.
- #1215 Fix YouTube iframes to be responsive (resolves unwanted side-effect of horizontal scrollbar at mobile/tablet media queries)
- #1208 Add description to
Plugin
definition and updateddiscovery.yml
and UI to consume it - #1195 Add temporary message in configuration communicating their global nature until "Profiles" are implemented
- #1245 Add detailed information on the documentation about events tracked by Meltano when Anonymous Usage Data tracking is enabled.
- #1228 Add preselections of the first column and aggregate of base table to initialize Analyze with data by default.
-
#1244 Add instructions on how to deactivate a virtual environment
-
#1082 Meltano will now enable automatically DAGs created in Airflow
-
#1231 Update CLI output during project initialization
-
#1126 Minor UI updates to improve clarity around Schedule step and Manual vs Orchestrated runs
-
#1210 Improved SQLite loader configuration context (name and description)
-
#1185 Remove majority of unimplemented placeholder UI buttons
-
#1166 Clarify in documentation that plugin configuration is stored in the
.meltano
directory, which is in.gitignore
. -
#1200 Link to new Getting Help documentation section instead of issue tracker where appropriate
-
#1227 Update Notebook
MainNav
link to jump to our Jupyter Notebook docs
-
#1075 Fix a bug that caused
target-csv
to fail. -
#1233 Fix the Design page failing to load a Design that has timeframes on the base table
-
#1187 Updated configuration to support
readonly
kind to prevent unwanted editing -
#1187 Updated configuration to setting resets to prevent unwanted editing
-
#1187 Updated configuration to conditionally reset certain settings to prevent unwanted editing
-
#1187 Updated configuration to prevent unwanted editing until we handle this properly with role-based access control
-
#1187 Updated certain connector configuration settings with a
readonly
flag to prevent unwanted editing in the UI. This is temporary and will be removed when we handle this properly with role-based access control. -
#1198 Fix "More Info." link in configuration to properly open a new tab via
target="_blank"
-
#1229 Improve extractor schema autodiscovery error messages and don't attempt autodiscovery when it is known to not be supported, like in the case of tap-gitlab
-
#1207 Updated all screenshots in Getting Started Guide to reflect the most current UI
- #991 Add e2e tests for simple sqlite-carbon workflow
- #1103 Add Intercom to Meltano.com to interact with our users in real-time
- #1130 Add Tutorial for extracting data from Google Analytics and loading the extracted data to Postgres
- #1168 Speedrun video added to home page and new release issue template
- #1182 Add
null
able date inputs so optional dates aren't incorrectly required in validation - #1169 Meltano now generates the dbt documentation automatically
- !1061 Update the Getting Started Guide and the Meltano.com documentation with the new UI and information about job logging and how to find the most recent run log of a pipeline.
- #1213 Add VuePress use and benefits to documentation
- #922 Document the importance of transformations and how to get started
- #1167 Iterate on docs to improve readability and content updates
- #1173 Fix
sortBy
drag-and-drop bug in Analyze by properly usingtryAutoRun
vs.runQuery
- #1079
meltano elt
will now run in isolation under.meltano/run/elt
- #1204 move project creation steps out of the local installation section of the docs and into the Getting Started Guide
- #782 Update timeframe label and fix timeframe attributes to properly display in the Result Table
- #1123 Add first-class "Submit Issue" CTA to help expedite resolution when a running job fails. Also updated the "Log" CTA in the Pipelines UI to reflect a failed state.
- #1172 Fix analytics issue related to app version
- #1121 Add ability to configure listen address of Meltano and Airflow
- #1022 Add "Autorun Query" toggle and persist the user's choice across sessions
- #1060 Auto advance to Job Log from Pipeline Schedule creation
- #1111 Auto advance to Loader installation step when an extractor lacks entity selection
- #1013 Toast initialization and analytics initialization cleanup
- #1050 Fix a bug where the Job log would be created before the
transform
are run. - #1122
meltano elt
will now properly run when usingtarget-snowflake
. - #1159 Minor UI fixes (proper
MainNav
Model icon active color during Analyze route match & "Run" auto query related cleanup) and...NameFromRoute
refactor renaming cleanup
- #1106 Add description metadata to the GitLab extractor's Ultimate License configuration setting
- #1057 Auto advance to Entity Selection when an extractor lacks configuration settings
- #51 Update Google Analytics to track
appVersion
, customprojectId
, and to properly use the defaultclientId
. The CLI also now usesclient_id
to differentiate between a CLI client id (not versioned) and the project id (versioned). - #1012 Add intelligent autofocus for improved UX in both Extractor and Loader configuration
- #758 Update 'meltano permissions' to add --full-refresh command to revoke all privileges prior to granting
- #1113 Update 'meltano permissions' to have the ability to find all schemas matching a partial name such as
snowplow_*
- #1114 Update 'meltano permissions' to include the OPERATE privilege for Snowflake warehouse
- Compress meltano-logo.png
- #1080 Temporarily disable Intercom until userId strategy is determined
- #1058 Updated the selected state of grouped buttons to fill vs. stroke. Updated the docs to reflect the reasoning to ensure consistency in Meltano's UI visual language
- #1068 Replace dogfooding term in docs to speedrun
- #1101 Add new tour video to home page
- #1101 Update design to improve readability and contrast
- #1115 Update 'meltano permissions' to not require an identially named role for a given user
- #1120 Fix a concurrency bug causing
meltano select
to crash. - #1086 Fix a concurrency issue when the
meltano.yml
file was updated. - #1112 Fix the "Run" button to improve UX by properly reflecting the running state for auto-running queries
- #1023 Fix last vuex mutation warning with editable
localConfiguration
clone approach
- Patch technicality due to PyPi limitation (v1 already existed from a publish mistake seven+ months ago) with needed changelog New/Changes/Fixes section headers
- #1020 Update Command Line Tools documentation to reflect a standard format with opportunities for improvement in the future
- #524 There is a new Plugins section on the site to contain all ecosystem related libraries (i.e., extractors, loaders, etc.)
- #1087 Fix
meltano select
not seeding the database when run as the first command. - #1090 Update the namespace for all plugins. Also the default schema used will go back to including the
tap_
prefix to avoid conflicts with existing schemas (e.g. a localgitlab
orsalesforce
schema). This also fixestap-csv
andtap-google-analytics
not properly working after the latest Meltano release. - #1047 Fix a bug where some configuration values were not redacted
- #1085 Fix Analyze model dropdown to properly reflect installed
models
- #1089 Properly re-initialize the Analyze page after a new analysis is selected during an existing analysis (this issue surfaced due to the recent Analyze dropdown CTAs addition which enables an analysis change during an existing one)
- #1092 Fix async condition so the design store's
defaultState
is properly applied before loading a new design viainitializeDesign
- #51 Add Google Analytics tracking acknowledgment in the UI
- #926 Add step-by-step intructions for using the DigitalOcean one-click installer
- #1076 Enable Log button in pipelines UI after route change or hard refresh if a matching log exists
- #1067 Add Model landing page and update Analyze main navigation to a dropdown displaying the various analysis CTAs associated with each model
- #1080 Add live chat support on Meltano.com website using Intercom.io
- #1069 Meltano will now use the schedule's name to run incremental jobs
- #926 Move manual DigitalOcean Droplet configuration instructions to advanced tutorials
- Collapse Installation docs into a single section
- #1071 Fix
rehydratePollers
so the UI reflects running jobs after a hard refresh or route change (this surfaced from the recent !963 change) - #1075 Fix an issue where
meltano elt
would fail when a previous job was found
-
#950 Removed the Analyze connection configuration: Meltano will now infer connections out of each loader configuration.
-
#1002 Analyze UI now displays the Topic's (analysis model's) description text if applicable
-
#1032 Add 'Model' and 'Notebook' to main navigation to communicate that Meltano plans to empower users with modeling and notebooking functionality
-
#949 Add "Log" button and dedicated sub-UI for tracking an ELT run's status more granularly
-
#932 Meltano can now be upgraded from the UI directly.
- #1045 Make it clear that 'meltano add' is not hanging while installing plugins
- #1000 Update Getting Started guide with updated screenshots and content
- #854 Charts now use pretty labels rather than the ID
- #1011 Removed "Catch-up Date" in favor of default "Start Date" of extractor
- #578 Remove support for
tap-zuora
. - #1002 Update
discovery.yml
with explicitkind: password
metadata (we infer and set input types ofpassword
as a safeguard, but the explicit setting is preferred) - #1049 Change default
target-sqlite
database name towarehouse
to not conflict with system database - #949 Update the way Meltano handles logs for ELT runs: Every elt run is logged in
.meltano/run/logs/{job_id}/elt_{timestamp}.log
. That allows Meltano to keep logs for multiple, or even concurrent, elt runs with the samejob_id
. - #949 Update "Create Pipeline" redirect logic based on the previous route being 'transforms' (this is a UX win setting up the user with the sub-UI for the next logical step vs. requiring a manual "Create" click)
- #1051 Automatically set SQLALCHEMY_DATABASE_URI config to system database URI
- #1004 Fix error when deselecting last attribute in Analyze
- #1048 Fix various actions that should have been mutations and did minor code convention cleanup
- #1063 Fix the "Explore" button link in Dashboards to properly account for the
namespace
- #1051 Remove MELTANO_BACKEND e.a. in favor of --uri CLI option and MELTANO_DATABASE_URI env var
- #1052 Move system database into
.meltano
directory to indicate it is owned by the app and not supposed to be messed with directly by users
- #1014 Meltano now logs all output from each
meltano elt
run in a log file that uses the unique job*id of the run. It can be found in.meltano/run/logs/elt*{job_id}.log
. - #1014 Meltano now logs all output from each
meltano elt
run in a log file that uses the unique job*id of the run. It can be found in.meltano/run/logs/elt*{job_id}.log
. - #1014 Meltano now logs all output from each
meltano elt
run in a log file that uses the uniquejob_id
of the run. It can be found in.meltano/run/logs/elt*{job_id}.log
. - #955 Establish baseline for demo day and how they should be run
- #891 Contributors can run webapp from root directory
- #1005 Fix installed plugins endpoints listing identically named plugins of different types under wrong type
- #987 Update routing to match labels (verbs vs. nouns) in effort to subtly reinforce action taking vs. solely "thing" management
- #960 Improve UX by instantly displaying extractor and loader configuration UIs based on "Install" or "Configure" interaction as opposed to the prior delay (side effect of async
addPlugin
) - #996 Update conditional UI analytics stats tracking at runtime vs. build-time by sourcing state from the same backend
send_anonymous_usage_stats
flag
- #992 Fix missing GA scripts
- #989 Fix UI/UX documentation regarding recent removal of
view-header
- #994 Fix stale Pipelines Count in main navigation Pipeline badge
- #999 Update yarn dependencies to resolve peer dependency warning
- #1008 Fix error on "Create Pipeline Schedule" modal when no plugins have been installed
- #1015 Support SQLite database name with and without '.db' extension
- #1007 Fix pipeline with failed job not being regarded as having completed
- #998 Update Analyze UI with conditional loading indicator to prevent query generation prior to connection dialects being loaded (this solution is still useful for when inference supercedes our current manual dialect selection solution)
- #1009 Fix default ConnectorSettings validation to account for
false
(unchecked) checkbox values
- #976 Route changes will update page title in the web app
- Marketing #48 Update newsletter subscription links to redirect to our new newsletter hosted by Substack
- #965 Fix a regression that prevented the Meltano UI to reach the Meltano API when using an external hostname.
- #986 Fix an issue where the Orchestration page would not show Airflow even when it was installed.
- #969 Fix an issue where the Meltano Analyze connection would not respect the
port
configuration. - #964 Fix copy button overlap issue with top navigation
- #970 Fix Meltano's m5o parser and compiler to properly namespace and isolate the definitions of different custom and packaged Topics.
- #980 Add Cypress for e2e testing pipeline
- #579 Add
meltano schedule list
to show a project's schedules - #942 Add progress bars on various routes to improve UX feedback
- #779 Add various UI polish details regarding iconography use, preloading feedback, breadcrumbs, container styling, navigation, and sub-navigation
-
#906
meltano ui
will now run inproduction
per default -
#942 Update Analyze Connections UI to match configuration-as-modal pattern for UX consistency regarding configuration
-
#779 Update all "This feature is queued..." temporary UI buttons to link to the Meltano repo issues page with a contextual search term
-
#927 Document how to manually set up a Meltano Droplet on DigitalOcean
-
#916 Add Transform step as first-class and adjacent step to Extract and Load
-
#916 Improve Create Pipeline Schedule default selection UX by leveraging "ELT recents" concept
-
#936 Add "Refresh Airflow" button in Orchestrate to bypass route change or full-page refresh when iframe doesn't initially inflate as expected (this will likely be automated once the root cause is determined)
-
#899 Add deep linking improvements to reports and dashboards to better facilitate sharing
-
#899 Add "Edit" and "Explore" buttons to each report instance displayed in a dashboard to enable editing said report and exploring a fresh and unselected analysis of the same model and design
-
!546 Add new Advanced Tutorial on how to Load CSV files to Postgres
- #909 Default names will be generated for Reports and Dashboards
- #892 Improve experience for parsing Snowflake URL for ID by showing processing step
- #935 Update Entity Selection to be nested in the Extract step so each ELT step is consecutive
- #886 Add validation for grouping settings as the next iteration of improved form validation for generated connector settings
- #931 Fix Analyze Connections identifier mismatch resulting from recent linting refactor
- #919 Fix Airflow iframe automatic UI refresh
- #937 Fix Chart.vue prop type error
- #838 Add indicator for speed run plugins
- #870 Add global footer component in docs
- #871 Add contributing link in footer of docs
- #908 Add auto installation for Airflow Orchestrator for improved UX
- #912 Auto run the ELT of a saved Pipeline Schedule by default
- #907 Add auto select of "All" for Entities Selection step and removed the performance warning (a future iteration will address the "Recommended" implementation and the display of a resulting performance warning when "All" is selected and "Recommended" ignored)
- #799 Standardized code conventions on the frontend and updated related documentation (issues related to further linting enforcement will soon follow)
- #838 Speed run plugins prioritized to top of the list
- #896 Add documentation for how to do patch releases
- #910 Update linting rules to enforce better standards for the frontend code base
- #885 Add docs for all extractors and loaders
- #885 All plugin modal cards show docs text if they have docs
- #733 Improve error feedback to be more specific when plugin installation errors occur
- #923 Fix contributing release docs merge conflict issue
- #746 Add CTA to specific dashboard in "Add to Dashboard" sub-UI
- #746 Add toast feedback on success, update, or error for schedules, reports, and dashboards
- #814 Install Airflow via the Orchestration UI (we may do this in the background automatically in the future)
- #901 Update entities plugins to be alphabetically sorted for consistency with extractors ordering
- #746 Prevent duplicate schedule, report, and dashboard creation if there is an existing item
- #976 Fix fallback v976e Route changes will update page title in the web appfor Iso8601 dates/times
- #903 Fix columns display issue for the base table in Analyze
- #894 Fix issue with static asset paths
- #894 Fix build issues with new Vue CLI 3 build process
- #763 Add inference to auto install related plugins after a user installs a specific extractor
- #867 Add fallback values (if they aren't set in the
discovery.yml
) forstart date
,start time
, andend date
for all connectors so the user has potentially one less interaction to make per connector configuration
- #342 Swap UI app directory "webapp" and upgrade to Vue CLI 3
- #882 Update navigation and subnavigation labels to verbs vs. nouns to inspire action and productivity when using the UI
- #700 Update documentation to remove "$" and trim spaces to make CLI command copy/paste easier
- #878 Write a tutorial to help users get started with PostgreSQL
- #883 Break Extractors and Loaders sections out in the docs
- #889 Allow for githooks to lint on commit
- #835 Pipeline name in Schedule creation will have an automatic default
- #872 Updated
tap-marketo
andtap-stripe
to leverage password input type while also improving the input type password fallback - #882 Fix recent minor regression regarding
Dashboard
routing - #858 Fix
job_state
bug so that ELT run status polling can properly resume as expected - #890 Fix implementation of default configuration setting to use less code
- #793 Add introduction module to Connector Settings to allow for helper text as far as signup and documentation links
- #796 Add dropdown option to Connector Settings to allow for more defined UI interactions
- #802 Add support for Query Filters over columns that are not selected
- #855 Add empty state to Dashboards and cleaned up styling for consistency with Analyze's layout
- #856 Add contextual information to the Analyze Connection UI to aid user understanding
- #800 Add save success feedback for connectors, entities, and connections
- #817 Add Meltano explainer video to the front page of Meltano.com
- #794 Update Snowflake fields to have descriptions and utilize tooltip UI
- #853 Improve UX for multi-attribute ordering (wider sub-UI for easier reading, clear drop target, and clearer drag animation for reenforcing sorting interaction)
- #735 Update Entities UI to only display entity selection "Configure" CTAs for installed (vs. previously all) extractors
- #548 Update Meltano mission, vision and path to v1 on roadmap page of Meltano.com
- #824 Update
meltano select
to use the uniquetap_stream_id
instead of thestream
property for filtering streams. This adds support for taps with multiple streams with the same name, like, for example, the ones produced bytap-postgres
when tables with the same name are defined in different schemas. - #842 Collapse Deployment section in the docs to be under Installation
- #855 Fix bug that duplicated a dashboard's
reportIds
that also prevented immediate UI feedback when reports were toggled (added or removed) from a dashboard via Analyze's "Add to Dashboard" dropdown - #851 Fix report saving and loading to work with filters and sortBy ordering
- #852 Update Scheduling UI to have "Run" button at all times vs conditionally to empower users to run one-off ELT pipelines even if Airflow is installed
- #852 Update Scheduling UI "Interval" column with CTA to install Airflow while communicating why via tooltip
- #852 Fix initial Orchestration page hydration to properly reflect Airflow installation status
- #831 Update
meltano elt
to exit with 1 and report dbt's exit code on an error message when dbt exits with a non-zero code. - #857 Update PluginDiscoveryService to use the cached
discovery.yml
when Meltano can not connect tomeltano.com
while trying to fetch a fresh version of the discovery file. - #850 Fix entities response so entities display as expected (as assumed this simple fix was due to our recent interceptor upgrade)
- #800 Fix connector and connection settings to display saved settings by default while falling back and setting defaults if applicable
- !781 Add new Advanced Tutorial on how to use tap-postgres with Meltano
- #784 Add multiple attribute ordering with drag and drop ordering in the UI
- #784 As part of multiple attribute sorting and keeping the attributes and results sub-UIs in sync, we know autorun queries based on user interaction after the initial explicit "Run" button interaction
- #821 Fix
meltano config
not properly loading settings defined in themeltano.yml
- #841 Fix a problem when model names were mangled by the API
- #834 Fixed a problem with the Meltano UI not having the proper API URL set
- #757 Update 'meltano permissions' to add support for GRANT ALL and FUTURE GRANTS on tables in schemas
- #760 Update 'meltano permissions' to add support for granting permissions on VIEWs
- #812
meltano ui
will now stop stale Airflow workers when starting - #762 Added run ELT via the UI (manages multiple and simultaneous runs)
- #232 Meltano now bundles Alembic migrations to support graceful database upgrades
- #828 Docker installation instructions have been dogfooded, clarified, and moved to Installation section
- #944 Update the Transform step's default to "Skip"
- #807 Fix filter input validation when editing saved filters
- #822 Fix pipeline schedule naming via slugify to align with Airflow DAG naming requirements
- #820 Fix
meltano select
not properly connecting to the system database - #787 Fix results sorting to support join tables
- #832 Fix schedule creation endpoint to return properly typed response (this became an issue as a result of our recent case conversion interceptor)
- #819 Running the Meltano UI using gunicorn will properly update the system database
- #788 Reydrate filters in Analyze UI after loading a saved report containing filters
- #804 Connection set in the Design view are now persistent by Design
- #788 Properly reset the default state of the Analyze UI so stale results aren't displayed during a new analysis
- !806 Fix filters editing to prevent input for
is_null
andis_not_null
while also ensuring edits to existing filter expressions types adhere to the same preventitive input. - #582 Remove the
export
statements in the default.env
initialized bymeltano init
. - #816 Fix
meltano install
failing when connections where specified in themeltano.yml
- #786 Fixed an issue with the SQL engine would mixup table names with join/design names
- #808 Fix filter aggregate value with enforced number via
getQueryPayloadFromDesign()
asinput type="number"
only informs input keyboards on mobile, and does not enforce the Number type as expected
- #759 Added filtering functionality to the Analyze UI while additionally cleaning it up from a UI/UX lens
- #792 Fix an error when trying to schedule an extractor that didn't expose a
start_date
.
- !718 Add support for filters (WHERE and HAVING clauses) to MeltanoQuery and Meltano's SQL generation engine
- #748 Added the
Connections
plugin to move the Analyze connection settings to the system database - #748 Added the
meltano config
command to manipulate a plugin's configuration
!726 Fixed InputDateIso8601's default value to align with HTML's expected empty string default
- #766 Add Codeowners file so that the "approvers" section on MRs is more useful for contributors
- #750 Various UX updates (mostly tooltips) to make the configuration UI for scheduling orchestration easier to understand
- #739 Updated
discovery.yml
for better consistency of UI order within each connector's settings (authentication -> contextual -> start/end dates). Improved various settings'kind
,label
, anddescription
. Added adocumentation
prop to provide a documentation link for involved settings (temp until we have better first class support for more complex setting types)
- #737 Fixed UI flash for connector settings when installation is complete but
configSettings
has yet to be set - #751 Fixed the Orchestrations view by properly checking if Airflow is installed so the correct directions display to the user
- #736 Add "Cancel", "Next", and a message to the entities UI when an extractor doesn't support discovery and thus entity selection
- #730 Updated Analyze Models page UI with improved content organization so it is easier to use
- #710 Updated connector (extractor and loader) settings with specific control type (text, password, email, boolean, and date) per setting, added form validation, and added an inference by default for password and token fields as a protective measure
- #719 Added InputDateIso8601.vue component to standardize date inputs in the UI while ensuring the model data remains in Iso8601 format on the frontend.
- #643 Updated
minimallyValidated
computeds so that new users are intentionally funneled through the pipelines ELT setup UI (previously they could skip past required steps) - #752 Fix the schedule having no start_date when the extractor didn't expose a
start_date
setting
- !703 Fix
ScheduleService
instantiation due to signature refactor
- #724 Add the
model-gitlab-ultimate
plugin to Meltano. It includes .m5o files for analyzing data available for Gitlab Ultimate or Gitlab.com Gold accounts (e.g. Epics, Epic Issues, etc) fetched using the Gitlab API. Repository used: https://gitlab.com/meltano/model-gitlab-ultimate - #723 Add proper signage and dedicated sub-navigation area in views/pages. Standardized the view -> sub-view markup relationships for consistent layout. Directory refactoring for improved organization.
- #612 Move the plugins' configuration to the database, enabling configuration from the UI
- #636 Refactored connector logo related logic into a ConnectorLogo component for code cleanliness, reusability, and standardization
- #728 Change error notification button link to open the bugs issue template
- #718 Fix dynamically disabled transforms always running. Transforms can now be dynamically disabled inside a dbt package and Meltano will respect that. It will also respect you and your time.
- #684 Enables WAL on SQLite to handle concurrent processes gracefully
- #732 Fix plugin installation progress bar that wasn't updating upon installation completion
- !683 Add
--start-date
tomeltano schedule
to give the control over the catch up logic to the users - #651 Added model installation in the Analyze UI to bypass an otherwise "back to the CLI step"
- #676 Add pipeline schedule UI for viewing and saving pipeline schedules for downstream use by Airflow/Orchestration
- #708 Enable
tap-gitlab
to run using Gitlab Ultimate and Gitlab.com Gold accounts and extract Epics and Epic Issues. - #711 Add new call to action for submitting an issue on docs site
- #717 Enable
dbt-tap-gitlab
to run using Gitlab Ultimate and Gitlab.com Gold accounts and generate transformed tables that depend on Epics and Epic Issues.
- #716 Fix entities UI so only installed extractors can edit selections
- #715 Remove reimport of Bulma in
/orchestration
route to fix borked styling
- !640 Google Analytics logo addition for recent tap-google-analytics Extractor addition
- #671 Add the
tap-google-analytics
transform to Meltano. It is using the dbt package defined in https://gitlab.com/meltano/dbt-tap-google-analytics - #672 Add the
model-google-analytics
plugin to Meltano. It includes .m5o files for analyzing data fetched from the Google Analytics Reporting API. Repository used: https://gitlab.com/meltano/model-google-analytics - #687 Implemented a killswitch to prevent undefined behaviors when a Meltano project is not compatible with the installed
meltano
version
- #661 Fixed empty UI for extractors that lack configuration settings by providing feedback message with actionable next steps
- #663 Fixed Airflow error when advancing to Orchestration step after installing and saving a Loader configuration
- #254 Fixed
meltano init
not working on terminal with cp1252 encoding - #254 Fixed
meltano add/install
crashing on Windows - #664 Minor CSS fix ensuring Airflow UI height is usable (side-effect of recent reparenting)
- #679 Fix an issue with
meltano select
emitting duplicate properties when the property used theanyOf
type - #650 Add
MELTANO_DISABLE_TRACKING
environment variable to disable all tracking - #670 Update tests to not send tracking events
- #603
meltano select
now supports raw JSON Schema as a valid Catalog - #537 Add Extractor for Google Analytics (
tap-google-analytics
) to Meltano. It uses the tap defined in https://gitlab.com/meltano/tap-google-analytics/
- #645 Fixed confusion around Loader Settings and Analytics DB Connector Settings
- #580 Fixed
project_compiler
so the Analyze page can properly display custom topics - #658 Fixed the Analyze page when no models are present
- #603 Fix an issue where
meltano select
would incorrectly report properties as excluded - #603 Fix an issue where
meltano select
incorrectly flatten nested properties - #553 Fix an issue where running
meltano select --list
for the first time would incorrectly report properties
- #586
meltano ui
now automatically start Airflow if installed; Airflow UI available atOrchestration
. - #592 Added baseline UX feedback via toast for uncaught API response errors with a link to "Submit Bug"
- #642 Improved UX during extractor plugin installation so settings can be configured during installation as opposed to waiting for the (typically lengthy) install to complete
- !647 Added preloader for occasional lengthy extractor loading and added feedback for lengthy entities loading
- #645 Added an Analyze landing page to facilitate future sub-UIs including the Analyze database settings; Added proper Loader Settings UI.
- #645 Fixed confusion around Loader Settings and Analyze database settings
- #622 Added ELT flow UI Routes & Deep Linking to advance user through next steps after each step's save condition vs. requiring them to manually click the next step to advance
- #598 Updated color and greyscale use in the context of navigation and interactive elements to better communicate UI hierarchy
- #607 Add "All/Default/Custom" button bar UI for improved entities selection UX
- #32 Integrate Algolia Search for docs
- #590 Add documentation for deploying Meltano in ECS
- #628 Add documentation for tap-mongodb
- !605 Added tooltips for areas of UI that are WIP for better communication of a feature's status
- 375 Meltano can now run on any host/port
- #595 Fix
meltano invoke
not working properly withdbt
- #606 Fix
SingerRunner.bookmark_state()
to properly handle and store the state output from Targets as defined in the Singer.io Spec.
- #32 Integrate Algolia Search for docs
- #522 Update Carbon tutorial with new instructions and screenshots
- #477 Add ability for users to sign up for email newsletters
- !580 Add sorting to plugins for improved UX, both UI via extractors/loaders/etc. and
meltano discover all
benefit from sorted results - !528 Add documentation for RBAC alpha feature and environment variables
- #588 Updated core navigation and depth hierarchy styling to facilitate main user flow and improved information architecture
- #591 Revert #484: remove
meltano ui
being run outside a Meltano project. - #584 Initial v1 for enabling user to setup ELT linearly through the UI via a guided sequence of steps
- #600 Fix a bug with meltano select when the extractor would output an invalid schema
- #597 Automatically open the browser when
meltano ui
is run
- #477 Add ability for users to sign up for email newsletters
- #591 Revert #484: remove
meltano ui
being run outside a Meltano project.
- Add documentation on custom transformations and models. Link to Tutorial: https://www.meltano.com/tutorials/create-custom-transforms-and-models.html
- #539 Add Tutorial for "Using Jupyter Notebooks" with Meltano
- #534 Add UI entity selection for a given extractor
- #520 Add v1 UI for extractor connector settings
- #486 Add the
model-gitlab
plugin to Meltano. It includes .m5o files for analyzing data fetched using the Gitlab API. Repository used: https://gitlab.com/meltano/model-gitlab - #500 Add the
model-stripe
plugin to Meltano. It includes .m5o files for analyzing data fetched using the Stripe API. Repository used: https://gitlab.com/meltano/model-stripe - #440 Add the
model-zuora
plugin to Meltano. It includes .m5o files for analyzing data fetched using the Zuora API. Repository used: https://gitlab.com/meltano/model-zuora - #541 Add a 404 page for missing routes on the web app
- #576 Fix switching between designs now works
- #555 Fix
meltano discover
improperly displaying plugins - #530 Fix query generation for star schemas
- #575 Move Airflow configuration to .meltano/run/airflow
- #571 Fix various routing and API endpoint issues related to recent
projects
addition
- #513 Added initial e2e tests for the UI
- #431 Add the
tap-zendesk
transform to Meltano. It is using the dbt package defined in https://gitlab.com/meltano/dbt-tap-zendesk - 484 Updated
meltano ui
to automatically launch the UI, and projects from the UI (previously only an option in the CLI) - #327 Add
meltano add --custom
switch to enable integration of custom plugins - #540 Add CHANGELOG link in intro section of the docs
- #431 Add the
model-zendesk
plugin to Meltano. It includes .m5o files for analyzing data fetched using the Zendesk API. Repository used: https://gitlab.com/meltano/model-zendesk - !544 Add support for extracting data from CSV files by adding tap-csv to Meltano
- #514 Add 'airflow' orchestrators plugin to enable scheduling
- Add the
tap-zuora
transform to Meltano. It is using the dbt package defined in https://gitlab.com/meltano/dbt-tap-zuora
- #455 Add documentation about
target-snowflake
- #507 Ensure design name and table name don't need to match so multiple designs can leverage a single base table
- #551 Fix HDA queries generated when an attribute is used both as a column and as an aggregate.
- #559 Add support for running custom transforms for taps without default dbt transforms.
- #432 Add the
tap-zuora
transform to Meltano. It is using the dbt package defined in https://gitlab.com/meltano/dbt-tap-zuora
- Remove Snowflake references from advanced tutorial.
- #2 dbt-tap-zuora Remove custom SFDC related attributes from Zuora Account and Subscription Models
- Update Contributing - Code Style documentation to including pycache troubleshooting
- #529 Resolve "SFDC Tutorial - ELT Fails due to invalid schema.yml" by #4 dbt-tap-salesforce removing the schema.yml files from the dbt models for tap-salesforce.
- #502 Fix the situation where an m5o has no joins, the design still will work.
- #485 Added various UI unit tests to the Analyze page
- #370 Enabled authorization using role-based access control for Designs and Reports
- #283 Silence pip's output when there is not error
- #468 Added reminder in docs regarding the need for
source venv/bin/activate
in various situations and added minor copy updates
- #433 Add the
sandbox
configuration totap-zuora
. - #501 Fix
meltano ui
crashing when the OS ran out of file watcher. - #510 Fix an issue when finding the current Meltano project in a multi-threaded environment.
- #494 Improved documentation around tutorials and Meltano requirements
- #492 A few small contextual additions to help streamline the release process
- #503 Fix a frontend sorting issue so the backend can properly generate an up-to-date query
- Add support for extracting data from Gitlab through the updated tap-gitlab (https://gitlab.com/meltano/tap-gitlab)
- Add the
tap-gitlab
transform to Meltano. It is using the dbt package defined in https://gitlab.com/meltano/dbt-tap-gitlab - Add "Copy to Clipboard" functionality to code block snippets in the documentation
- Add the
tap-stripe
transform to Meltano. It is using the dbt package defined in https://gitlab.com/meltano/dbt-tap-stripe - Add new command
meltano add model [name_of_model]
- Add models to the available plugins
- Various documentation installation and tutorial improvements
- Added troubleshooting button to help users add context to a pre-filled bug issue
- Fix the API database being mislocated
- Replaced the stale Meltano UI example image in the Carbon Emissions tutorial
- 473: Fix the docker image (meltano/meltano) from failing to expose the API
- locks down dependencies for issues with sqlalchemy snowflake connector
- Add Salesforce Tutorial to the docs
- Add documentation for the permissions command
- Add tracking for the
meltano ui
command
- Updated analytics to properly recognize SPA route changes as pageview changes
- Update stages table style in docs
- Add custom transforms and models tutorial to the docs
- Add api/v1 to every route
- Update DbtService to always include the my_meltano_project model when transform runs
- Resolved duplicate display issue of Dashboards and Reports on the Files page
- Removed legacy
carbon.dashboard.m5o
(regression from merge) - Updated dashboards and reports to use UI-friendly name vs slugified name
- Fix minor clipped display issue of right panel on
/settings/database
- Fix minor display spacing in left panel of Settings
- Fix dashboard page to properly display a previously active dashboard's updated reports
- Fix pre-selected selections for join aggregates when loading a report
- Fix charts to display multiple aggregates (v1)
- Fix 404 errors when refreshing the frontend
- Fix a regression where the Topics would not be shown in the Files page
- Add the
tap-salesforce
transform to Meltano. It is using the dbt package defined in https://gitlab.com/meltano/dbt-tap-salesforce - Add m5o model and tables for tap-salesforce
- Updated the deep-link icon (for Dashboards/Reports on the Files page)
- Polished the RBAC view, making it clearer the feature is experimental.
- Rename "Models" to "Topics"
- Use the current connection's schema when generating queries at run time for Postgres Connections.
- Add support for multiple Aggregates over the same attribute when generating HDA queries.
- UI cleanup across routes (Analyze focus) and baseline polish to mitigate "that looks off comments"
- Update installation and contributing docs
- Meltano implement role-based access control - !368
- Add version CLI commands for checking current Meltano version
- Add deep linking to dashboards
- Add deep linking to reports
- Fixed a problem when environment variables where used as default values for the CLI - !390
- Fixed dashboards initial load issue due to legacy (and empty)
carbon.dashboard.m5o
file - New standardized approach for
.m5o
id generation (will need to remove any dashboard.m5o and report.m5o)
- Update installation and contributing docs
- Add support for generating Hyper Dimensional Aggregates (HDA)
- Add internal Meltano classes for representing and managing Designs, Table, Column, Aggregate, Definitions, and Query definitions
- Move core functionality out of
api/controllers
to/core/m5o
(for m5o and m5oc management) and/core/sql
(for anything related to sql generation)
- Fixed a problem when environment variables where used as default values for the CLI - !390
- Add gunicorn support for Meltano UI as a WSGI application - !377
- Meltano will now generate the minimal joins when building SQL queries - !382
- Add analytics to authentication page
- Meltano will now use SQLite for the job log. See https://meltano.com/docs/architecture.html#job-logging for more details.
- Removed manual
source .env
step in favor of it running automatically
- Meltano will correctly source the
.env
- fixed charts to render as previously they were blank
- Fixed Analyze button groupd CSS to align as a single row
- Meltano will now use SQLite for the job log. See https://meltano.com/docs/architecture.html#job-logging for more details.
- URL routing updates ('/model' to '/files', removed currently unused '/extract', '/load', '/transform' and '/project/new')
- add ability to save reports
- add ability to update an active report during analysis
- add ability to load reports
- add dashboards page and related add/remove report functionality
- Generate default
Meltano UI
connection for themeltano.db
SQLite DB when a new project is created withmeltano init
- updated main navigation to Files, Analysis, and Dashboards
- Update the
meltano permissions grant
command to fetch the existing permissions from the Snowflake server and only return sql commands for permissions not already assigned - Add
--diff
option to themeltano permissions grant
command to get a full diff with the permissions already assigned and new ones that must be assigned
- Entry model definition correctly defines
region_id
. - Updated the Fundamentals documentation section regarding reports
- Fixed Files page for empty state of Dashboards and Reports
- Fixed Analyze page's left column to accurately preselect columns and aggregates after loading a report
- Add tracking of anonymous
meltano cli
usage stats to Meltano's Google Analytics Account - Add
project_config.yml
to all meltano projects to store concent for anonymous usage tracking and the project's UUID
- Add
--no_usage_stats
option tomeltano init <project_name>
to allow users to opt-out from anonymous usage stats tracking - Bundled Meltano models are now SQLite compatible.
- Added basic authentication support for meltano ui.
- Meltano will now automatically source the .env
- Updated docs with
.m5o
authoring requirements and examples - add support for timeframes in tables
- add basic analytics to understand usage
- add disabled UI for the lack of timeframes support in sqlite
- update Results vs. SQL UI focus based on a results response or query update respectively
- Meltano will now discover components based on
https://meltano.com/discovery.yml
- sample designs are now packaged with meltano
- Updated mobile menu to work as expected
- Updated tutorial docs with improved CLI commands and fixed the host setting to
localhost
- add new command
meltano add transform [name_of_dbt_transformation]
- add transforms to the available plugins
- Auto install missing plugins when
meltano elt
runs - Terminology updates for simpler understanding
- Edit links on the bottom of doc pages are working now
- Updated docs tutorial bullet regarding inaccurate "Validate" button
- ensure
meltano init <project-name>
runs on windows - settings ui now provides sqlite-specific controls for sqlite dialect
- add
target-sqlite
to available loaders for meltano projects - add new command
meltano add transformer [name_of_plugin]
- add transformers (dbt) to the available plugins
- extractors and loaders are arguments in the elt command instead of options
meltano www
is nowmeltano ui
- remove dbt installation from
meltano init
- move everything dbt related under
transform/
- update
meltano elt
to not run transforms by default - update
meltano elt
to auto generate the job_id (job_id has been converted to an optional argument)
- left joins now work correctly in analyze.
- fixed broken sql toggles in analyze view
- fixed sql output based on sql toggles in analyze view
- add Using Superset with Meltano documentation
- add default models for 'tap-carbon-intensity'.
- Meltano Analyze is now part of the package.
- removes database dependency from Meltano Analyze and uses .ma files
- update the error message when using Meltano from outside a project - !238
- updated Settings view so each database connection can be independently disconnected
- add
meltano select
to manage what is extracted by a tap.
-
documentation site will utilize a new static site generation tool called VuePress
-
meltano.com will be deployed from the meltano repo
- model dropdown now updates when updating database (no longer requires page refresh)
- prevent model duplication that previously occurred after subsequent "Update Database" clicks
- documentation site will utilize a new static site generation tool called VuePress
- first iteration of joins (working on a small scale)
- resolve version conflict for
idna==2.7
- fix the
discover
command in the docker images - fix the
add
command in the docker images - fix module not found for meltano.core.permissions.utils
- add
meltano permissions grant
command for generating permission queries for Postgres and Snowflake - !90 - add 'tap-stripe' to the discovery
- demo with carbon intensity, no API keys needed
- .ma file extension WIP as alternative to lkml
- fix order in Meltano Analyze
- add default values for the 'www' command - !185
- add CHANGELOG.md
- fix a problem with autodiscovery on taps - !180
- move the 'api' extra package into the default package
- add 'tap-fastly' to the discovery
- remove
setuptools>=40
dependency meltano
CLI is now in themeltano
package
- target output state is now saved asynchronously
- initial release