Skip to content

Releases: dolthub/doltgresql

0.17.1

17 Feb 17:08
@github-actions github-actions
v0.17.1
3cc7a54
Compare
Choose a tag to compare
0.17.1 Latest
Latest

Merged PRs

doltgresql

  • 1206: Added loop structures and label support
    This adds the rest of the "non-FOR" loops, along with loop control flow and labels. I also noticed that we weren't using OperationSizeForStatements, which will properly calculate offsets for the Goto operations, so I fixed that. Just so happened that our tests weren't running into the issue.
  • 1200: Fix backup restore auth for super user
    Dolt PR: dolthub/dolt#8860
  • 1199: NULL support in PL/pgSQL
    We get support for NULL statements from pg_query_go since it automatically filters them out. This change adds a test of their usage.
  • 1197: WHILE statements in PL/pgSQL
    Adds basic support for WHILE statements in PL/pgSQL. No support for labels yet.
  • 1196: Fixed CREATE FUNCTION not executing some statements
    Simple bug, but was difficult to spot. Some statements work without needing to advance the returned RowIter, while others require advancing it. I changed it so that we're always advancing the iterator. I also unskipped the skipped tests, and removed the temporary one since the original now works.
  • 1193: Added SELECT ... INTO and PERFORM, plus bugfixes
    Hoped to get a few more statements in, but there's an issue where some statements that are ran from the interpreter are seemingly being discarded. It was a roadblock even for these two, hence the skipped tests. That's the next thing I'mma look into, as it's a pretty big issue (especially since there's no error, just no effect which is even worse).
  • 1171: Bug fixes for using various types in keys
    This PR allows various types to be used in keys that were not possible before. This comes down to two things:
    1. Swap out the validateCreateTable analyzer rule
    2. Change the determination for the max width of a type to be more accurate, which influences whether we choose standard extended encoding or extended address encoding
    3. Alter the way we determine the encoding type for extended columns in keys to more closely match the non-key case (dolthub/dolt#8817)
      This PR also has a couple small regressions in tests. I'm punting on these because we're about to completely overhaul how we store these types (to implement toast semantics).

Closed Issues

Assets 8
Loading

0.17.0

11 Feb 21:57
@github-actions github-actions
v0.17.0
b62b735
Compare
Choose a tag to compare
0.17.0

Merged PRs

doltgresql

  • 1194: Fix SSP linker errors for Windows build
    Pulling in the pg_query_go library added a CGO dependency, and the Windows build requires the stack smash protection library to be linked in. Unix builds do not, because it's built into Clang.
  • 1183: Bug fix: Function variable reference substitution
    In function bodies, when replacing variable names with variable reference IDs, we were replacing any text matching the variable name, which wasn't always correct. This new approach tokenizes the expression and replaces only the tokens that exactly match a known variable name. There is still at least one more edge case we can fix in a follow up.
  • 1177: Can now call created functions
    This makes a simple change so that we're now able to call user-created functions. This is intended as a temporary measure, as we ultimately want functions to be tied to commits (similar to how sequences and types are currently handled). That also entails changing how function calls are altogether handled, so rather than doing the larger change now, this is a stop-gap so that we can continue making progress on CREATE FUNCTION.
  • 1174: Parse CREATE FUNCTION into runnable functions
    This implements parsing of CREATE FUNCTION into the same expected set of operations as the handcoded examples. This means that we are able to write CREATE FUNCTION statements that are runnable, and the interpreter is able to properly execute said functions.
    At the moment, the function provider used in GMS does not allow for adding new functions, so the test is skipped. I've confirmed it does work though if we hack around the provider limitation. It's also worth noting that we have to rely on the new pg_analyze package in order to handle PL/pgSQL. I attempted to add support within our parser, but the syntax conflicts with regular SQL syntax in ways that can't really be reconciled, so this was the only choice.
  • 1169: PL/pgSQL: Add support for the ALIAS statement
    Adds support for the execution logic for the ALIAS statement in user defined functions.
  • 1158: Add an interpreter base for function creation
    This adds the base of an interpreter, upon which CREATE FUNCTION and CREATE PROCEDURE will convert their contents to. Only the interpreter portion is partially implemented, along with a temporary example function and accompanying test to ensure that it works.
    The next major step is to add parsing support for all of the statements that we'll need, as they do not yet exist (hence why the temporary example function's operations were manually created, although the conversion should result in the same output). The last major step will be to convert from the PL/pgSQL code to the operations that our interpreter expects.
    The original intention was to implement as much of this in GMS as possible, however for now, the bulk of it will exist in Doltgres as I couldn't quite reconcile the proper way to implement it there without diving more fully into MySQL's CREATE FUNCTION and CREATE PROCEDURE support (rather than simply relying on memory). This is something that I'll revisit in a later PR, as I feel it'll be easier to do once I have a relatively complete implementation done in Doltgres.
    Related PR:
  • 1157: Fixed several bugs related to column default expressions and generated columns
    Fixes a parser bug that disallowed calling a function with no arguments.
    Implements a missing form of the substring() function.
    Diff includes #1154
  • 1154: Env var to ignore all unsupported DDL operations / commands
  • 1152: bug fix for column defaults with function values
    Removed the GMS validateColumnDefaults rule in favor of our own, and also made it run on CreateTable statements (it wasn't before due to an oversight in a type assertion).
    Also implemented the gen_random_uuid function.
  • 1145: use cockroach errors.Errorf for one-off errors
    This PR changes all uses of fmt.Errorf() to use the equivalent cockroach library instead. This gives us stack traces for all errors constructed this way during testing and debugging.
  • 1143: Tests for various column default values
    Bug fix for new tests in dolthub/go-mysql-server#2825
  • 1120: Support for parsing VACUUM statements, as a no-op
  • 1113: Implementing new methods from servercfg.DoltgresConfig
    Implements the new IsUserSpecified() and SkipRootUserInitialization() methods from servercfg.DoltgresConfig, added in dolthub/dolt#8690
  • 1108: Wrap function expressions for generated columns in parens
  • 1107: Created wrappers around Internal IDs for consistency and added an ID dispatch registry
    Currently, an Internal ID requires one to know the exact format for constructing an ID of a specific type. For example, Internal table IDs have the following format: 
    [ Schema_Name, Table_Name ]
    Not only would you need to know that this is the format, but when retrieving values, you'd have to make sure that you accessed index 0 for the schema, and index 1 for the table. This opens up a potential for hard-to-find bugs. Wrappers help with this. Internally, the wrappers will call index 0 or 1, but one now just calls SchemaName() or TableName(). In addition, they don't need to know the format, calling NewInternalTable gives you the parameters that you need.
    Additionally, this adds a registry where schema elements may register callbacks to respond to ID changes. For example, rather than having a table column explicitly handle all locations that depend on the column, it instead broadcasts that the column has changed, and all locations that have registered for that broadcast will modify themselves. This helps with scaling, as Postgres is far more inter-dependent than MySQL/Dolt, and reduces the knowledge burden required by developers.
    Currently it's only used by sequences in this PR, and that's primarily due to most actions being handed off to GMS and Doltgres having no record of them. For example, deleting a column is completely handled by GMS, but Doltgres needs to broadcast that the column was deleted. For now, the idea is to wrap all such nodes in a Doltgres node at the end of the analyzer step (in ReplaceNode), but I'm open to another idea since it essentially means we're making a wrapper for the majority of GMS nodes. That's a separate issue from this PR though.
  • 1103: Implemented a no-op node for statements that can be safely ignored
    Used for OWNER ddl statements for now, and will extend to VACUUM next.
  • 1097: CliContext interface update. NoOp
    Account for interface change in Dolt: dolthub/dolt#8708
  • 1095: Better support for COPY FROM statements
    This change makes it possible to load data into tables without the full set of columns specified. pg_dump will not include generated columns in its output.
    Also fixes a bug for the load from file use case, which only supported CSV files before (now supports the same set of formats).

Closed Issues

Loading

0.16.0

30 Dec 20:41
@github-actions github-actions
v0.16.0
0ae6752
Compare
Choose a tag to compare
0.16.0

Merged PRs

doltgresql

  • 1086: Implement metrics, describe system table tests
    GMS PR: dolthub/go-mysql-server#2800
  • 1083: Go tests for the COPY command
    These are all lifted from the bats tests. I'm gearing up to make major changes to how COPY works to support more customer schemas and want to these tests to help.
  • 1081: support enum type cast
  • 1077: select from functions
    Depends on: dolthub/go-mysql-server#2795
  • 1076: Added Internal ID representation
    This adds an internal ID that is used instead of OIDs.
  • 1069: Update CONTRIBUTING.md
    The unicode char at the end of the request for stars wasn't rendering. replaced with a ⭐
  • 1065: Removed ownership, added DROP privilege
    This removes ownership and related information, so that now users must be granted appropriate privileges by either a superuser or another user with the appropriate WITH GRANT OPTION permission. In addition, this adds a brand new DROP privilege, so that there's now a way to allow non-superusers to delete tables.
  • 1064: Add skipped test for where clause bug, README fix
  • 1055: Removed unused config params
  • 1045: support create shell and enum types
    This PR adds support for CREATE TYPE for ENUM and SHELL types. It adds only parsing support for COMPOSITE type, functionality support for it is coming after it.
  • 1044: Lots of bug fixes discovered in engine testing
    Replaces #1015
    Includes bug fixes for:
    • Certain limit/offset queries
    • Parenthesized table expressions
    • Table renames
    • Better harness support
  • 1042: .github: Some small workflow fixes
  • 1040: Release v0.15.0
    Created by the Release workflow to update DoltgreSQL's version
  • 1015: Fixed bug in certain limit .. offset queries
  • 997: support explicit cast to domain type
    Regressed tests are tables used as type for casting, which was not supported. The insert statement that makes use of it fails currently. It will be supported as an extension after upcoming CREATE TYPE composite type support.
  • 987: reapply DoltgresType changes with performance improvement
    Regressed CREATE VIEW test is tracked here.
  • 969: Add support for pg_get_serial_sequence(text,text)
    Adds support for the pg_get_serial_sequence(text,text) function, which returns the sequence owned by a column.
    Example usage: 
    create table t1 (id SERIAL PRIMARY KEY);
SELECT pg_get_serial_sequence('t1', 'id');

Closed Issues

  • 464: Support SELECT * FROM function() syntax
  • 735: Panic when querying dolt_diff after updating a text field
Loading

0.15.0

05 Dec 22:11
@github-actions github-actions
v0.15.0
4155b93
Compare
Choose a tag to compare
0.15.0

Merged PRs

doltgresql

  • 1035: Tests and schema fixes for dolt.status, dolt.merge_status and dolt_constraint_violations_*
  • 1033: switch DoltgresType to be used as pointer
  • 1029: Fix schemas for dolt system tables with boolean columns
    Dolt PR: dolthub/dolt#8634
  • 1014: Reworked DoltgresType function access
    This reworks how functions are accessed within DoltgresType, in addition to adding a faster path for functions that fall under specific criteria.
  • 1004: Add tests for dolt_statistics
    GMS: dolthub/go-mysql-server#2762
    Dolt: dolthub/dolt#8589
  • 1003: Add diff tests for dolt.docs
  • 998: implement to_char for timestamp (mostly)
    This PR adds support for most of the date format specifiers mentioned here: https://www.postgresql.org/docs/current/functions-formatting.html
  • 996: Support dolt.rebase, add tests
  • 995: Tests for dolt_ignore per schema
    Dolt PR: dolthub/dolt#8581
  • 994: Tests and fixes for dolt_rebase and dolt_ignore
    Dolt PR: https://github.com/dolthub/dolt/pull/8578/files
  • 988: Added a mini sysbench test that runs on PRs
    This adds a new workflow that runs on every PR, similar to the regression tests. This runs a small version of the sysbench tests, primarily to detect large performance changes that a PR may unexpectedly introduce. For now, the variance is set to ±10%. That is, if it's under 10% in either direction, it'll just display as a white color. If it's greater, then it'll display as green or red respectively to indicate a possible performance regression. There's variability in the tests, that we'll want to account for, but perhaps the 10% window is too large. Or maybe we should run the tests for longer.
  • 986: Revert "Merge pull request #904 from dolthub/jennifer/type"
    This reverts commit 40dba4f, reversing changes made to 165c30e.
    Reverting this change because of performance regression.
  • 983: Major improvements to auth
    This adds a myriad of tests, in addition to:
    • Schema-Wide Table Privileges
    • Schema Privileges
    • Database Privileges
    • Role Memberships
      There are also many smaller changes, such as disallowing non-superusers from modifying superusers, and many more.
  • 981: support insert DEFAULT VALUES
    This is postgres special syntax that allows you to just insert all default values.
    This is equivalent to insert into <tbl> values () in MySQL.
  • 979: support EXCEPT set operation
  • 978: support EXISTS subquery
    This PR adds support for EXISTS <subquery> clause.
    Additionally, we have to add a bunch of skips namely in the setup queries in the testing/go/regression/results directory; these queries encounter performance issues for joins over pg_catalog tables discussed here: #940
  • 977: update Regression test to display 50 tests each at most with total number of tests
  • 970: catch more unsupported partition of queries
    These were panicking before, but now we throw unsupported
  • 965: Fix DEFAULT insert for NULL or unspecified column default values
  • 964: Add dolt_schema and dolt_procedure tests
  • 963: TestUpdate and TestDelete
  • 949: Release v0.14.1
    Created by the Release workflow to update DoltgreSQL's version
  • 945: support with
    somewhat related: #928
  • 904: create DoltgresType struct used for all types
    This PR replaces current DoltgresType interface with DoltgresType struct. All built-in types will have pre-defined values for this struct. User-defined types will populate the struct fields with appropriate values.
    Depends on dolthub/go-mysql-server#2736

Closed Issues

  • 985: pull requests need #benchmark so latency regressions can be caught before they're merged.
  • 943: Unable to set statement_timeout
  • 937: doltgres 0.14.0 regression test failure
  • 843: Error on restore into doltgres (pg_restore)
Loading

0.14.1

13 Nov 00:15
@github-actions github-actions
v0.14.1
1a5e206
Compare
Choose a tag to compare
0.14.1

Merged PRs

doltgresql

  • 947: Fix parameter context for statement_timeout and recovery_target_name
    The statement_timeout parameter had the wrong context configured, which prevented users from being able to change its value. I did a quick skim of Doltgres' parameters and compared them to pg_catalog.pg_settings and found one other parameter context (recovery_target_name) that didn't match.
    Fixes: #943
  • 942: Removed doltgres user and default database
    This removes the default doltgres user, so that we're more in-line with a standard Postgres installation. In addition, this also removes the default doltgres database, and fully replaces it with a postgres database. Postgres creates this database by default, so it's best that we mimic it.
    In addition, Postgres uses the username as the database name if a database name is not given, and I'm sure many users simply use the postgres user without even thinking about what database they're connecting to, so this allows for that workflow to work as expected.
  • 941: throw unsupported error for PARTITION OF statements
    Catch PARTITION OF statements and return error instead of panicking.
  • 940: Add caching for pg_catalog tables
    While investigating performance on some skipped tests for pg_catalog tables, I found a few areas to optimize around how we deal with OIDs. The biggest source of latency I found was that pg_catalog tables must iterate through every object in a database (e.g. schemas, tables, indexes, types) to generate OIDs and create the rows for each pg_catalog table. When multiple pg_catalog tables are joined together, this starts to consume a significant amount of time, with some queries triggering this database iteration process hundreds of thousands of times.
    This change creates a new cache in the session object that stores the data for pg_catalog tables so that it can be reused within the same query. The effect is that several queries that timed out before will now complete. Some of these queries are still slow (e.g. multiple seconds to execute) and can be further optimized. The biggest source of latency now seems to be from the join operations, since the data is currently not eligible for point lookups and has to be scanned repeatedly to stitch rows together.
    Another area for future optimization is the regclass/regtype/regproc IO_Output and IO_Input functions, which also trigger iterations over database objects to turn a relation name into an OID, and to turn an OID into a relation name.
    Longer term, instead of relying on iterating database objects and caching OID information, we eventually need to store this information so it can be quickly loaded, and keep it in sync as database objects are created/modified/deleted, including changes made through merges, resets, and other database version control changes.
  • 939: Unskip many InsertInto engine tests
    This adds several new features / bug fixes found during testing:
    • DROP INDEX support
    • Support for ON CONFLICT UPDATE
    • Panic during insert of some UNION statements
    • Fixed case sensitivity bug in column names for INSERT statements
  • 938: parse explain select and explain options
  • 936: Release v0.14.0
    Created by the Release workflow to update DoltgreSQL's version
  • 935: implement multiple table inherits
    This PR supports most of the functionality around PostgreSQL INHERITs.
    We just translate create table t1 (...) inherits (t2, t3, ...); to create table t1 like (t2, t3, ...);
    Note: MySQL does not support multiple LIKE statements
    gms logic: dolthub/go-mysql-server#2738
    syntax: dolthub/vitess#375

Closed Issues

  • 943: Unable to set statement_timeout
Loading

0.14.0

08 Nov 05:45
@github-actions github-actions
v0.14.0
3a470e8
Compare
Choose a tag to compare
0.14.0

Merged PRs

doltgresql

  • 934: support VALUES()
    Add support for select ... from (values (...), (...), ...) sqa statements
  • 921: more partition no-ops
  • 919: support multi-alter statements
    We support multi alter statements, so we shouldn't error.
  • 918: partitions are no-ops
    We parse, but ignore, PARTITION BY clauses in CREATE TABLE statements in gms, so do the same thing here.
    Docs noting this here: dolthub/docs#2392
  • 909: have rootFinalizerIter implement sql.MutableRowIter
    We've moved rules out of analyzer into FinalizeIters so the applyRowUpdateAccumulator is applied afterwards.
    However, this seems to be a special sql.RowIter, so we need to expose the child iterators through the sql.CustomRowIter interface to apply that rule
  • 908: Tests for system tables with _$tablename
    Dolt PR: dolthub/dolt#8527
  • 901: More system table updates, add and unskip some tests
  • 898: more unskipped engine tests
  • 887: Bug fix: Make db.Ping(ctx) work correctly with Doltgres
    Implementations of db.Ping(ctx) from Go's database/sql package typically send some sort of empty query to test the liveness of a database connection. For example, the pq library sends ; and the pgx library sends -- ping.
    Before this change, Doltgres' parser would return an empty statement, instead of Vitess' ErrEmpty error, which caused GMS to try and execute a nil analyzed plan for these statements, return an error, and the ping check would fail. This change makes the Doltgres parser return the same Vitess ErrEmpty error that the Vitess parser throws, sending the same signal to GMS to handle an empty statement without returning an error to the client. The related PR dolthub/go-mysql-server#2716 updates the Parser interface documentation to explicitly call out the requirement to return ErrEmpty in order for empty statements to be handled correctly.
    For testing, I've added some no-op statements to the smoke tests, but since these tests go through as prepared statements, they don't follow the exact code path as db.Ping(ctx), so I've also added calls to db.Ping(ctx) in the test framework, which do reproduce this error. I've also added a unit test for Doltgres' Parser implementation that explicitly tests empty statement parsing.
    Fixes: #884
    Related to: dolthub/go-mysql-server#2716
  • 886: /testing/logictest/harness/doltgres_server_harness.go: skip db.Ping call
  • 882: drop domain support
    Added:
    • merge functionality for domain types
    • check for domain usage in tables for dropping domain
      TODO:
    • creating domain type creates instance of array type of this domain type
    • unique OID generated for each domain created
    • add domains to pg_type
    • ALTER TABLE (e.g.: modify existing column to domain type column)
    • use domain type as underlying type to another domain
  • 881: Update first batch of dolt system tables to use dolt schema
    • dolt_status -> dolt.status
    • dolt_branches -> dolt.branches
    • dolt_log -> dolt.log
    • dolt_tags -> dolt.tags
    • dolt_docs -> dolt.docs
      These changes are mostly backwards compatible, unless you have an existing doc in dolt_docs, which will need to be recreated after updating
  • 879: Allow nextval() to take a regclass instance
    Allows the nextval() function to take a regclass parameter. Also changes the output function of regclass so that the returned relation name is schema-qualified if the schema is not on search_path.
    This change also moves the GetCurrentSchemas() function into a new settings package to break a package import cycle. The new settings package is intended for low-level functions that access settings, without dependencies on other packages.
    Fixes: #850
  • 878: tests for cross-schema foreign keys
    Depends on:
    dolthub/go-mysql-server#2713
    dolthub/dolt#8479
  • 877: Function: set_config()
    Adds support for the set_config() function.
    Example Usage: 
    SELECT set_config('mynamespace.foo', 'bar', false);
set_config
------------
bar
SELECT current_setting('mynamespace.foo');
current_setting
-----------------
bar
    Fixes: #852
  • 872: rule changes
  • 868: More ALTER TABLE Support
    Adds support for additional ALTER TABLE syntax:
    • Adding a UNIQUE constraint
    • Adding a CHECK constraint
    • Dropping constraints
      Adding support for check constraints triggered an issue with string literal value quoting that affects check constraints and column defaults. The fix was to make expression.Literal.String() match the behavior of GMS' expression.Literal.String() method and quote string literals. This required fixing another spot where we had been adding in quotes for string literals, as well as a small change in GMS (dolthub/go-mysql-server#2710).
      Fixes:
    • #799
    • #800
       
      Regresions Report:
      The regressions listed below are a little tricky to read, but everything seems to be working correctly as far as I can tell. In the first regression listed (INSERT INTO inhg VALUES ('foo');), this query now fails, because a previous statement now executes correctly to add a check constraint to a table, but our CREATE TABLE LIKE logic incorrectly copies over check constraints.
      The rest of the regressions listed seem to actually be working correctly and I'm unable to repro problems with them, and they aren't reporting any errors in the regression report. For example, I've confirmed that the regression reported for ALTER TABLE inhx add constraint foo CHECK (xx = 'text'); actually executes correctly without error now, while on main it returns the error: ALTER TABLE with unsupported constraint definition type *tree.AlterTableAddConstraint.
  • 867: Release v0.13.0
    Created by the Release workflow to update DoltgreSQL's version
  • 863: Added GRANT, REVOKE, Privilege Checking, Ownership, and persisting changes
    This adds the majority of the core functionality that was missing for users, authorization, and privilege checking. This is missing rigorous testing, but that will be added in a separate PR (both engine and bats). All changes that will accompany those tests (such as missing statement support) will also be added in separate PRs.
  • 845: add create domain and domain usage support
    Depends on: dolthub/go-mysql-server#2697
    TODO:
    • add merge functionality for domains
    • creating domain type creates instance of array type of this domain type
    • unique OID generated for each domain created
    • track domain usage in table for dropping domain
    • add domains to pg_type
    • ALTER TABLE (e.g.: modify existing column to domain type column)
    • use domain type as underlying type to another domain

Closed Issues

  • 884: db.Ping results in error
  • 850: Column default value expression error
  • 852: Support set_config() function
  • 799: Support ALTER TABLE ... DROP CONSTRAINT
  • 800: Support adding unique constraints through ALTER TABLE
Loading

0.13.0

17 Oct 21:13
@github-actions github-actions
v0.13.0
8deb190
Compare
Choose a tag to compare
0.13.0

Merged PRs

doltgresql

  • 862: Foreign key tests
    Depends on dolthub/dolt#8461
  • 858: Fix merge bug, add tests
    Fixes #855
    Dolt PR: dolthub/dolt#8459
  • 856: fix type for coalesce regression test
  • 851: Bug fixes for double-quoted relation names
    While investigating #843, several statements were failing due to double-quoted identifiers. This PR fixes COPY statements and use of nextval() and setval() to properly parse double-quoted identifiers.
  • 834: Label contribution PRs
    Adds a GitHub Actions workflow to label PRs from outside the team with the "contribution" label. This is the same workflow we use for dolt/go-mysql-server/etc.
  • 833: Test and fix for detached head error for pg_catalog tables
    Dolt PR: dolthub/dolt#8434
  • 832: Bug fixes for DESCRIBE and implemented SHOW TABLES
    Fixes #830
  • 829: Support for using TEXT fields in secondary indexes and UNIQUE constraints
    Allows TEXT columns in Doltgres to work when used in secondary indexes and in unique constraints. PostgreSQL does not have a concept of prefix lengths and Dolt requires prefix lengths for TEXT columns. This now works in Doltgres because a new Doltgres analyzer rule implicitly fills in a default prefix length in these cases.
    Depends on:
  • 828: Support for DESCRIBE with AS OF
  • 824: Add information_schema tests for checkout and detached head
  • 820: Unskip tests for dolt_revert
    Stacked on #818
  • 818: Support for per-table AS OF clauses
  • 816: Unskip COPY ... DELIMITER test
  • 815: Tests for dolt_history table
  • 813: Added CREATE USER, ALTER USER, DROP USER
    This adds support for CREATE USER, ALTER USER, and DROP USER. This also adds the related ROLE statements too, since the USER statements are essentially aliases for them. Next step will be to look into persistence and bats testing.
  • 812: fix: fix slice init length
    The intention here should be to initialize a slice with a capacity of length rather than initializing the length of this slice.
    The online demo: https://go.dev/play/p/q1BcVCmvidW
  • 809: Unskip dolt_patch test
    Dolt PR: dolthub/dolt#8415
  • 804: add column check constraint support for create table
    depends on: dolthub/go-mysql-server#2683
  • 801: Fix diff functions, return schema.table for table_name columns
    Dolt PR: dolthub/dolt#8411
  • 790: Support for setting custom session settings in user namespaces
    e.g.: 
    SET mynamespace.var to 'true';
SHOW mynamespace.var;
SELECT current_setting('mynamespace.var');
    Postgres allows any configuration parameter to be set this way as long as it's not in the default (empty) namespace.
    This functionality lets us exercise Dolt engine tests that have user variables in them.
  • 789: Add support for ALTER COLUMN TYPE, DEFAULT, and NOT NULL statements
    Depends on:
  • 788: Diff tests for changes in multiple schemas
  • 787: add functions translate and quote_ident and add pg_operator table
    Also:
    • Removed pg_stat_io view as it is supported for Postgres versions 16 and later.
    • Fixed parsing date type input - fixes #773
  • 786: Add diff tests for single schema
  • 785: ALTER TABLE support for ADD COLUMN, DROP COLUMN, and RENAME COLUMN
  • 783: Test fixes for merge and related operations
  • 782: Release v0.12.0
    Created by the Release workflow to update DoltgreSQL's version
  • 779: enable using postgres unix socket file path
  • 778: Feature: delimiter support for COPY FROM STDIN
    Allows the DELIMITER copy option to be specified for CSV and TEXT imports.
    Also adds initial support for the legacy copy option syntax described in the Compatibility section of COPY docs (e.g. COPY <table> FROM STDIN CSV, HEADER; instead of COPY <table> FROM STDIN WITH (FORMAT CSV, HEADER true);).
  • 775: Added RFC5802 for user client authentication
    For the most part, this implements the following portions of RFC 5802:
    • Section 2.2
    • Section 3
    • Validation portion of Section 5
      This also bases the stored data from:
    • https://www.postgresql.org/docs/15/catalog-pg-authid.html
      This PR finally lets the server do the full authentication routine with the client. For now, I've created a mock database full of mock users, which are actually being tested by all of our engine tests since we do supply a username and password when creating and setting up the server. The pgx library is handling the client-side authentication for these tests.
      The next step is to handle basic CREATE USER and DROP USER statements. With those in, I'll create a full battery of tests (bats tests, unit tests, dedicated engine tests, etc.). Unit tests are not included in this PR since I may make further changes with the next PR, and I'd prefer to do a big testing pass at once since none of this is in production just yet.
      After the aforementioned statements and testing, I'll move on to designing the storage portion, since the mock user and mock database are stand-ins for an actual implementation.
  • 746: fix value types used for datetime types and add timezone function

    • Closed Issues

    • 855: dolt_merge stomps working set
    • 844: Not picking up a commit?
    • 830: DESCRIBE [table] does not work for tables in other schemas
    • 822: information_schema tables do not work in detached head state
    • 808: Support DELIMITER in COPY...FROM
    • 33: Default socket file is named mysql.sock
    • 773: Support PostgreSQL datetime literal formats
    • 728: Support socket interface
Loading

0.12.0

30 Sep 22:50
@github-actions github-actions
v0.12.0
dbf3557
Compare
Choose a tag to compare
0.12.0

Merged PRs

doltgresql

  • 780: Don't count failure to send standby messages as terminal failures dur…
    …ing replication
  • 770: Tests for dolt_schemas
    Dolt PR: dolthub/dolt#8401
  • 768: Add workbench tests for docs
    Dolt PR: dolthub/dolt#8398
  • 766: More workbench tests for schema, tags, views
  • 763: Wrap column default expressions in ParenExprto match MySQL's requirements
    PostgreSQL's syntax does not require column default expressions to be wrapped in parens, but MySQL's does, so when we translate the column default value expressions to the vitess AST, we need to wrap them in parens so that they execute in GMS without triggering an error.
    Fixes #751
  • 762: Add support for ALTER TABLE ADD FOREIGN KEY
    Helps with data imports, since it's common to add FKs at the end of an import script.
    The regression below seems to come from the FK being added, but we don't yet support removing an FK, so the drop table call now fails.
    Related to #724
  • 761: Implement information_schema views and tables, add as schema
    GMS: dolthub/go-mysql-server#2678
    Dolt: dolthub/dolt#8391
  • 752: More unskipped tests
    Better test harness allowing to unskip about half of the dolt merge tests (after various bug fixes in dolt, already in main)
  • 747: Smoke tests for dolt diff functions/tables
    Dolt PR: dolthub/dolt#8381
  • 742: Fixed panic for some index joins
    Fixes:
    • #730
      For some index joins, the analyzer will create a specific type of plan that creates MySQL ranges rather than Doltgres ranges. It appears as though there may be divergent branches for the join logic, so I attempted to look for the source of the divergence, however I came up short.
      For now, rather than chasing this down and delaying a PR (since Tim needs this fixed asap), we can pass the lookup to the internal Dolt table. This will return incorrect results in some situations, but it won't panic for now, so I'll follow up with a better search through GMS at a later date to merge the index join paths.
  • 741: Support automatic transaction management with COPY FROM STDIN
  • 740: Add workbench tests for COPY FROM
  • 739: Tests for unique indexes
  • 736: Expanding tests for the steps in the Doltgres Getting Started Guide
    Added more test coverage over the Doltgres Getting Started Guide and pulled them out into their own file.
  • 733: Initial Authentication Protocol
    This implements the initial portion of the authentication protocol.
    Postgres Reference Documentation:
    • https://www.postgresql.org/docs/15/protocol-flow.html
    • https://www.postgresql.org/docs/15/sasl-authentication.html
      Primarily, this implements SASL SCRAM-SHA-256, which appears to be the primary form of authentication used in modern Postgres. It has been built by following the RFC specification:
    • https://datatracker.ietf.org/doc/html/rfc5802
      There are no tests since the implementation is incomplete. It cannot truly be tested until we have passwords and such that it can verify against (as the results must be sent back to the client for verification, so it can't be faked), however I have tested it up through what has been written, and what exists works as it should.
      Surprisingly, there aren't any libraries that we could really leverage for this. Most SASL libraries don't implement SCRAM. The closest was the following:
    • https://codeberg.org/mellium/sasl
      However, I couldn't really find a way to integrate it using raw messages and the eventual Doltgres user backend, so this is all custom-written using the RFC as a guideline (along with capturing packets using the regression capture tool to ensure that Postgres follows the RFC's implementation). For now, the logic is hidden behind a bogus parameter check so that the static analyzer is happy, and the next step is to make a mock in-memory database of users and passwords so I can fully test the entire workflow.
  • 732: Initial support for ALTER TABLE, starting with adding a primary key
    Adding initial support for converting ALTER TABLE statements. This first iteration only supports ALTER TABLE t ADD PRIMARY KEY (...);.
    Related to #724
  • 731: fix return limited row issue
  • 727: Enabled use mydb/main without quoting and implemented the IF function
    Most of this PR is changes to the doltgres engine testing harness to make it pass more tests.
    • Correctly validates most DML statements
    • Matches commit hashes more reliably
    • Handles more types
      Also include parser support for unquoted db identifiers with a / in a USE statement, and implements the IF function (apparently a cockroach extension).
  • 722: Feature: COPY support for HEADER option
    Adds support for using the HEADER option in COPY statements.
    In this first iteration, we only support specifying HEADER or HEADER true. This form causes the tabular data loader and CSV data loader to skip over the initial, header line in import data. In addition to this form, COPY also supports a HEADER MATCH option, where the server asserts that the columns in the import data exactly match the name and the order of the columns in the destination table.
    (Note: this PR is based off of #700 to help split up the changes to make them easier to review)
  • 717: Moved enginetests to their own testing CI workflow, don't use -race
    This works around the problem described here:
    #718
  • 711: Fix Regression Tests Workflow
  • 707: use pgproto3 for handling server connection messages
  • 705: Workbench tests for log, merge, table details
  • 700: Feature: COPY FROM STDIN support for CSV files
    Support for loading data via COPY FROM STDIN using CSV data.
  • 699: Unskip some engine tests
    dolt_reset now works correctly
  • 694: Tests for version control operations on new schemas / tables in new schemas
    Code changes are all in Dolt:
    dolthub/dolt#8343
  • 691: Convert ANALYZE statements
    Adds support for converting Postgres' ANALYZE statement for a single table and running it through the GMS SQL engine. There are stills lots of unsupported options in Postgres' ANALYZE statement, but this change allows Doltgres to process the simplest form – where a single table is being analyzed.
    Since it's common to run ANALYZE at the end of data load scripts (example), this change is intended to make it easier to load dumps into Doltgres.
  • 689: COPY FROM STDIN support
    Adds support for COPY ... FROM STDIN. When copying from STDIN, the COPY FROM statement starts a multi-message flow between the server and client – the client will send COPY DATA messages until all the data has been sent, and then send a COPY DONE (or COPY FAIL) message to finalize the transfer and let the server process more queries.
    This PR adds a new TabularDataLoader type, with the idea that we can create a DataLoader interface for that when we extend this to add support for loading data from CSV data files, too.
    This PR also depends on a GMS change to allow us to create a new sql.Context instance: dolthub/go-mysql-server#2652
  • 686: Added COPY FROM
    This was pulled from:
    • #672
      Separating the COPY FROM portion from the regression tests.
  • 681: use ParameterObjectIDs in Parse message for binding variable types
    Note: the prepared tests don't utilize this field in the Parse message, so it needs to extract the binding value types from analyzed plan of the query.
  • 680: Prevent premature decay for IN tuple
    This removes IN prematurely decaying, since it's only necessary for index filters. To complement this, I've implemented SplitConjunction and SplitDisjunction, so that they're aware of Doltgres expression types. The GMS versions will see *pgexprs.GMSCast and do nothing, since we actually care about the child, but GMS is unaware of that.
  • [677](https://github...
Read more
Loading

0.11.1

21 Aug 21:35
@github-actions github-actions
v0.11.1
1c9b990
Compare
Choose a tag to compare
0.11.1

Merged PRs

doltgresql

  • 640: current_schema function return non "$user" schema
  • 636: add Postgres default access methods in pg_am table
    This allows displaying index information on TablePlus
  • 635: More bug fixes for subqueries
    Fixes column resolution problems when using subqueries in joins.
    Partially addresses #554
  • 632: Support for IN with a subquery
    Tuples won't work until we have a doltgres tuple type, but should work for everything else.
    New expression type is a combination of InTuple from Doltgres and InSubquery from GMS. GMS code couldn't really be reused usefully.
  • 631: Unskip some subquery tests, add more workbench tests, allow cross-database references for regclass
  • 624: Bug fix for subqueries
    The context finalizer node was implemented incorrectly in terms of the semantics required for subquery logic to function -- the top-level child was never being considered in the row-prepend process.
    IN clauses with subqueries still do not work, that's next.
  • 621: add test for non default schema qualified view
    depends on dolthub/go-mysql-server#2633
  • 620: Allow varchar columns as primary keys
    The problem: the varchar type in pgtypes was declared as Text, not Varchar. This triggered validation checks in both GMS and Dolt when such a column was used in a primary key. I originally tried adding additional index / length info in the CREATE TABLE transformation, but with the varchar type being interpreted as Text, there's no combination of params that can satisfy all the validation checks.
    The error before this fix is e.g. 
    "ERROR: blob/text column 't' used in key specification without a key length"
    Given that we have a lot of validation logic with key prefix lengths related to BLOB / TEXT types, it seems like this type really can't be declared as TEXT, but open to other ideas.
    Collation / charset support is still fake and will need to be fixed, but no worse than it was before.
  • 617: use unknown type for string literal
    This PR also adds some build in functions:
    • pg_get_indexdef - parses
    • pg_indexes_size - parses
    • pg_tables_size - parses
    • pg_total_relation_size - parses
  • 614: Implement information_schema.schemata
    GMS PR: dolthub/go-mysql-server#2632

Closed Issues

  • 41: TablePlus can Connect and run queries but does not show Table Metadata
  • 14: TablePlus connection kind of works
  • 16: show create table and show tables should not work
  • 64: CREATE SCHEMA not yet supported
  • 171: current_schema() not yet supported
  • 599: querying a schema qualified view from different schema fails
Loading

0.11.0

12 Aug 19:02
@github-actions github-actions
v0.11.0
730b7ff
Compare
Choose a tag to compare
0.11.0

Merged PRs

doltgresql

  • 596: Implemented DISCARD
    Fixes #591
  • 585: support INTERVAL type and AGE() function
  • 584: add more pg functions and extract() function
    Added temporary resolution for handling ENUM type as TEXT type until we support CREATE TYPE statement.
    Added functions:
    • extract - partially works
    • pg_get_function_identity_arguments - parses
    • pg_get_partkeydef - parses
    • pg_postmaster_start_time - parses
  • 562: Add partially implemented unnest function
  • 560: add some of pg_... functions
    Added functions:
    • array_to_string - works
    • pg_encoding_to_char - parses
    • pg_function_is_visible - parses
    • pg_get_functiondef - parses
    • pg_get_triggerdef - parses
    • pg_get_userbyid - parses
    • pg_get_viewdef - works
    • pg_table_is_visible - works
  • 552: Add pg_get_expr for nil values, pg_attrdef table
  • 550: unskip and add tests for querying pg_catalog table without schema defined
    Depends on dolthub/dolt#8174
  • 544: Implement pg_get_constraintdef function
  • 530: Fixed function resolution for NULLs
    We had a workaround to allow function resolution to handle NULL expressions (NULL values worked fine since they still resolved to a non-NULL type). This adds the necessary logic so that we no longer need the workaround, and the NULL expression is properly resolved.
  • 526: Implement information_schema.columns for doltgres
    Relies on GMS PR dolthub/go-mysql-server#2600
  • 523: Convert all dolt_ procedures to functions so they can return values
    This PR also adds basic support for a Composite type, which needs to be fully fleshed out. Its values are sql.Row and its mysql wire type is Tuple.
    This works, in that you can now run e.g. 
    select dolt_commit('-am', 'mymessage');
    I'm looking for feedback on this approach before I spend any more time polishing it up and adding more tests.
    We will also want to support this syntax: 
    select * from dolt_commit(...);
    For functions that return composite types, the first syntax will return a single formatted column result, while the latter will return a the same number of columns, in the same types, as the underlying composite type. The latter is probably how most people will use this functionality if they want to examine the result of these function calls. Support for that syntax will be in a future PR.
  • 520: Implement ANY/SOME expression
  • 519: Fix "char" and array types for system functions and tables
  • 517: Add format_type function
  • 512: Fix using table names in columns, add skipped test for pg_catalog query
  • 508: Add tests for information_schema.columns
    Relies on dolthub/go-mysql-server#2596
  • 506: Add byteacat to binary concat
  • 502: Add anytextcat and textanycat functions to binary concat operator
  • 501: Add regtype type
  • 500: Implement binary concat operator for text types
  • 497: Add comment information system functions
    These will return an empty string until we support comments
  • 495: Fix column name for functions
    Fixes #486
  • 493: Fix prepared statements for Name type
  • 492: Added regclass and regproc
    This adds regclass and regproc, in addition to everything needed to support their full functionality. The hashing approach we previously used for OIDs were not compatible as-is, and reverse hashing would require extensive modifications to track all side-effects (table renames, etc.) in addition to needing some way to resolve hash collisions, so an approach similar to the one I proposed in the OID document was used.
    OIDs are now treated as indexes into arrays, and all database elements (tables, views, etc.) are iterated in a deterministic way, such that we can imagine that all tables are just a single large array containing every table from every schema, all views are a single large array, etc. This let's us get a definitive OID for any database element, and also when given any OID, we are able to grab the relevant database element.
    This is done by splitting an OID's bits into three sections.
    • Section: This is a header that specifies the search space. For example, tables have their own search space since we can iterate over all of them using database.GetTableNames and database.GetTableInsensitive. Each item has its own section, since they all have their own search spaces.
    • SchemaIndex: This is the index of the relevant schema. We order schemas by sorting them by their names (ascending), so the index gives us exactly the schema to use.
    • DataIndex: This is the index of whatever data the section represents. Some elements exist across multiple elements, such as the set of all indexes. To represent an index for those, we sort all tables by their name (ascending) and iterate over each table's indexes as though it were a single contiguous array.
      The size of each bit section is malleable, so I've made some guesses at what sensible starting values may be. In addition, I've disabled the ability to save tables with regclass and regproc, even though Postgres officially supports them. Since we do not yet have a permanent OID solution, I felt that it wouldn't be wise to allow persisting such values to disk, since changing the OID algorithm would break all such databases.
  • 487: Add more information_schema tests
    Depends on dolthub/go-mysql-server#2590
  • 484: Fix version function so that it returns Postgres version
    Relies on GMS PR: dolthub/go-mysql-server#2588
  • 482: /.github/scripts/performance-benchmarking/get-postgres-doltgres-job-json.sh: add additional tests to results
  • 481: Added IN to indexes
    select_random_points is performing slower than expected, and this is due to IN not using indexes. This just adds IN to the index logic, so that select_random_points will now actually use indexes. This logic does not properly reflect how PostgreSQL should really behave, but currently none of the index logic behaves properly due to GMS requiring exact type matches, rather than relying on comparison operators, so there's no additional harm in adding this.
    No new tests are added, as we already have pre-existing tests for IN using indexes. They just didn't actually use the index, but still returned the correct result, and now they're using the index as well.
  • 475: Added function support for polymorphic types
    This adds the types anyelement and anynonarray, in addition to adding support for defining functions that use polymorphic types.
    https://www.postgresql.org/docs/15/extend-type-system.html#EXTEND-TYPES-POLYMORPHIC
    Besides the changes to function resolution, this also adds a new parameter so that functions can see which types are passed in. This also cleans up the function interface, so that function definitions are a little more consistent and slightly less prone to typos (such as changing function parameters to an array from a slice, so that the length must be defined).
  • 471: Release v0.10.0
    Created by the Release workflow to update DoltgreSQL's version
  • 461: Fixed quoting in enginetest queries
    Automatically fix quoting in engine test query strings
  • 440: Update get-postgres-doltgres-job-json.sh

Closed Issues

  • 591: Support DISCARD statements
  • 590: Support INTERVAL type
  • 513: Should be able to select from pg_catalog tables without specifying pg_catalog
  • 442: select and drop on tables with the same name as pg_catalog table name without schema qualifier refers to pg_catalog table by default
  • 281: using table-qualified column name in SELECT stmt doesn't work
  • 511: Should be able to specify table name in column in SELECT
  • 486: SELECT function() returned column name is incorrect
  • 443: Brok...
Read more
Loading