Commit Graph

1708 Commits

Author SHA1 Message Date
Lunny Xiao fabf0a2895
Fix bug 2023-07-13 16:11:06 +08:00
Lunny Xiao 791cfa4834
Fix test 2023-07-13 15:55:46 +08:00
Lunny Xiao 230342d41b
some improvements 2023-07-13 13:27:55 +08:00
Lunny Xiao abc324c944
Remove unused functions 2023-07-13 13:22:07 +08:00
Lunny Xiao 6a783823fa
some refactors 2023-07-13 13:20:34 +08:00
Lunny Xiao f33221df74 Fix question mark replacement on postgres (#2202)
Fix #2112

Reviewed-on: https://gitea.com/xorm/xorm/pulls/2202
2023-07-12 13:22:32 +00:00
takumin 722f1cc141 Generate coverage html (#2170)
Co-authored-by: Lunny Xiao <xiaolunwen@gmail.com>
Reviewed-on: https://gitea.com/xorm/xorm/pulls/2170
Co-authored-by: takumin <takumiiinn@gmail.com>
Co-committed-by: takumin <takumiiinn@gmail.com>
2023-07-12 13:01:34 +00:00
Lunny Xiao 3b53a5f847 upgrade dependencies (#2223)
Fix #2209

Reviewed-on: https://gitea.com/xorm/xorm/pulls/2223
2023-07-12 11:06:39 +00:00
洪坤安 73eee961cc Performance Optimization: reduce slice2Bean calling times of strings.ToLower and map creation times (#2255)
1. 优化性能:减少func (session *Session) slice2Bean方法中的strings.ToLower调用次数以及减少map创建和访问次数(见工单 https://gitea.com/xorm/xorm/issues/2243)

2. 新增SetDefaultJSONHandler方法,用于用户自行设置DefaultJSONHandler(见工单 https://gitea.com/xorm/xorm/issues/2129)

优化前:
加载耗时=16.079s,总记录数=4028940
加载耗时=15.775s,总记录数=4028940
加载耗时=15.946s,总记录数=4028940

优化后:
加载耗时=10.863s,总记录数=4028940
加载耗时=11.257s,总记录数=4028940
加载耗时=11.155s,总记录数=4028940

Co-authored-by: Lunny Xiao <xiaolunwen@gmail.com>
Reviewed-on: https://gitea.com/xorm/xorm/pulls/2255
Co-authored-by: 洪坤安 <wuzili1234@163.com>
Co-committed-by: 洪坤安 <wuzili1234@163.com>
2023-07-12 09:53:19 +00:00
martinvigg 55d9407390 mysql: add CHAIN, RANK to reserved word list (#2282)
Reference: https://dev.mysql.com/doc/refman/8.0/en/keywords.html
Co-authored-by: Martin Viggiano <martin.viggiano@stonebranch.com>
Co-authored-by: Lunny Xiao <xiaolunwen@gmail.com>
Reviewed-on: https://gitea.com/xorm/xorm/pulls/2282
Co-authored-by: martinvigg <martinvigg@noreply.gitea.com>
Co-committed-by: martinvigg <martinvigg@noreply.gitea.com>
2023-07-12 08:52:23 +00:00
Lunny Xiao f1f5e7cd1a Some refactor (#2292)
replace #2285

Reviewed-on: https://gitea.com/xorm/xorm/pulls/2292
2023-07-12 07:53:25 +00:00
Lunny Xiao 79a8bc804b Fix join problem (#2291)
Fix #2284

Reviewed-on: https://gitea.com/xorm/xorm/pulls/2291
2023-07-12 02:01:56 +00:00
brookechen 486c344ba3 In SQLite3, Sync doesn't support Modify Column:Error: near MODIFY: syntax error (#2267)
rebase

Co-authored-by: brookechen <brookechen@noreply.gitea.io>
Co-authored-by: brookechen <brookechen@tencent.com>
Co-authored-by: Lunny Xiao <xiaolunwen@gmail.com>
Reviewed-on: https://gitea.com/xorm/xorm/pulls/2267
Co-authored-by: brookechen <brookechen@noreply.gitea.com>
Co-committed-by: brookechen <brookechen@noreply.gitea.com>
2023-07-11 17:10:36 +00:00
flyingpigge 52b01ce67f chore: ignore unnecessary char type cast in GetColumns (#2278)
PostgreSQL有两种char类型:`character`和`"char"`: https://www.postgresql.org/docs/current/datatype-character.html.

`pg_class`里的`relkind`是`"char"`类型,所以`GetColumns`这里应该转成`::"char"`或者和本PR里请求的一样去掉转换。这样对于PostgreSQL以及其他兼容PostgreSQL的数据库容错性更好,在做比较时它们通常都会被隐式转换。
```sql
postgres=# select pg_typeof(relkind), pg_typeof('a'::char), pg_typeof('a'::"char") from pg_class limit 1;
 pg_typeof | pg_typeof | pg_typeof
-----------+-----------+-----------
 "char"    | character | "char"
(1 row)
```

Co-authored-by: August <pin@singularity-data.com>
Co-authored-by: Lunny Xiao <xiaolunwen@gmail.com>
Reviewed-on: https://gitea.com/xorm/xorm/pulls/2278
Co-authored-by: flyingpigge <flyingpigge@noreply.gitea.com>
Co-committed-by: flyingpigge <flyingpigge@noreply.gitea.com>
2023-07-03 09:15:45 +00:00
Lunny Xiao d29fe49933 Mysql support a new tag Collate (#2283)
Fix #237
Fix #2179

Reviewed-on: https://gitea.com/xorm/xorm/pulls/2283
2023-07-01 03:40:09 +00:00
Lunny Xiao 18f8e7a86c Default don't log warn for database extra columns when syncing (#2280)
Reviewed-on: https://gitea.com/xorm/xorm/pulls/2280
2023-06-23 15:00:31 +00:00
Lunny Xiao 068de8c0f8 Don't warn when database have extra columns which are not in struct (#2279)
Reviewed-on: https://gitea.com/xorm/xorm/pulls/2279
2023-06-23 08:48:37 +00:00
ccbhj 838a0d9bca ccbhj/fix_mysql_blob_shared_bytes (#2274)
Copy the sql.RawBytes when converting to []byte.
Fix issue https://gitea.com/xorm/xorm/issues/2273

Co-authored-by: Bingjia Chen <bingjia.chen@seamoney.com>
Reviewed-on: https://gitea.com/xorm/xorm/pulls/2274
Reviewed-by: Lunny Xiao <xiaolunwen@gmail.com>
Co-authored-by: ccbhj <ccbhj@noreply.gitea.com>
Co-committed-by: ccbhj <ccbhj@noreply.gitea.com>
2023-06-12 09:18:13 +00:00
Lunny Xiao caa8a029c6 some optimzation (#2272)
Reviewed-on: https://gitea.com/xorm/xorm/pulls/2272
2023-06-02 14:16:30 +00:00
LinkinStars cb851a2f95 Filter support passing context (#2200) (#2270)
```diff
// Filter is an interface to filter SQL
type Filter interface {
---	Do(sql string) string
+++	Do(ctx context.Context, sql string) string
}
```

### Adds a `Context` parameter to the `Do` method of the Filter interface.

Developers can rewrite SQL through the `Filter` `Do` method and **need to get the necessary data from the Context** to assist.

For example, get user information through `Context`, so that different users can use different tables. Another example is to get the flags through `Context` to add annotations to the SQL, and use the annotations to let the subsequent `DB-Proxy` to achieve read/write separation.

Reviewed-on: https://gitea.com/xorm/xorm/pulls/2270
Reviewed-by: Lunny Xiao <xiaolunwen@gmail.com>
Co-authored-by: LinkinStars <linkinstar@foxmail.com>
Co-committed-by: LinkinStars <linkinstar@foxmail.com>
2023-05-31 01:43:24 +00:00
Zettat123 57f7d69f1b Fix test-cockroach workflow (#2269)
The image has already specified the entrypoint so we don't need to call `/cockroach/cockroach` in `cmd`.
![image](/attachments/669496c1-5673-4609-abc7-f42846e4d479)

Reviewed-on: https://gitea.com/xorm/xorm/pulls/2269
Co-authored-by: Zettat123 <zettat123@gmail.com>
Co-committed-by: Zettat123 <zettat123@gmail.com>
2023-05-29 13:01:43 +00:00
Lunny Xiao 04d36cfa81 Use actions instead of drone (#2258)
Reviewed-on: https://gitea.com/xorm/xorm/pulls/2258
2023-05-25 18:15:09 +08:00
brookechen 190384b4cd AutoIncrement列带ID插入数据时没有Commit (#2264)
业务场景中,需要预留(1 ~ 100)的ID给系统规则使用。所以会先使用插入将AutoIncrement列的id偏移到一个特定的值(如:100),然后“带ID调用Insert插入系统规则”。 当带ID插入时,由于没有commit,会被rollback掉。

Reviewed-on: https://gitea.com/xorm/xorm/pulls/2264
Reviewed-by: Lunny Xiao <xiaolunwen@gmail.com>
Co-authored-by: brookechen <brookechen@noreply.gitea.io>
Co-committed-by: brookechen <brookechen@noreply.gitea.io>
2023-05-17 15:20:40 +08:00
Martin Viggiano e6907e9a62 oracle: Fix quotes on DropTableSQL function, add IsSequenceExist function. (#2265)
- Fixed quotes on drop table SQL statement.
- Implemented IsSequenceExists function for oracle dialect.

Reviewed-on: https://gitea.com/xorm/xorm/pulls/2265
Reviewed-by: Lunny Xiao <xiaolunwen@gmail.com>
Co-authored-by: Martin Viggiano <martinvigg@gmail.com>
Co-committed-by: Martin Viggiano <martinvigg@gmail.com>
2023-05-17 09:04:00 +08:00
Abei1uo 23be940bad Fix session insert interface slice commit panic(#2259) (#2260)
Fix session insert interface slice commit panic(#2259)
-- TestInsertMulti2InterfaceTransaction is the test case

Reviewed-on: https://gitea.com/xorm/xorm/pulls/2260
Co-authored-by: Abei1uo <2676567028@qq.com>
Co-committed-by: Abei1uo <2676567028@qq.com>
2023-05-12 14:58:38 +08:00
Alberto Garcia 94fcec7f65 parse timestamp with milliseconds in tz format (#2246)
This pr fixes a bug caused when a timestmap in TZ format with milliseconds (`2023-04-05T15:50:48.256816Z` or `2023-04-05T15:50:48.256816+08:00`) is being parsed.

- The bug is happens in the function `String2Time` in `convert/time.go`
- if the timestamp contains milliseconds a layout with the structure `2006-01-02 15:04:05.` is used to parse it
- This layout contains an space between the date and the hour, if the timestamp returned by the db contains a `T` the parser fails and an error is returned

This pr adds a check for the `T` and milliseconds in the timestamp, if both conditions are present the date is parsed using the RFC3339 layout

Added test cases to check timestamps in this format

Solves [#2244](https://gitea.com/xorm/xorm/issues/2244)

Reviewed-on: https://gitea.com/xorm/xorm/pulls/2246
Reviewed-by: Lunny Xiao <xiaolunwen@gmail.com>
Co-authored-by: Alberto Garcia <agarcia@circutor.com>
Co-committed-by: Alberto Garcia <agarcia@circutor.com>
2023-04-06 21:47:59 +08:00
datbeohbbh d485abba57 add `(*Engine) SetConnMaxIdleTime` (#2229)
issue: https://gitea.com/xorm/xorm/issues/2228
Co-authored-by: datbeohbbh <phongtomfapp@gmail.com>
Reviewed-on: https://gitea.com/xorm/xorm/pulls/2229
Reviewed-by: Lunny Xiao <xiaolunwen@gmail.com>
Co-authored-by: datbeohbbh <datbeohbbh@noreply.gitea.io>
Co-committed-by: datbeohbbh <datbeohbbh@noreply.gitea.io>
2023-03-06 18:55:33 +08:00
fanshengshuai 914f2db9ea mysql字段为UNSIGNED时,会导致字段类型无法识别,返回RawBytes,JSON 字段映射为String (#2225)
如题

Reviewed-on: https://gitea.com/xorm/xorm/pulls/2225
Reviewed-by: Lunny Xiao <xiaolunwen@gmail.com>
Co-authored-by: fanshengshuai <fanshengshuai@noreply.gitea.io>
Co-committed-by: fanshengshuai <fanshengshuai@noreply.gitea.io>
2023-02-28 23:42:42 +08:00
KN4CK3R 056cecc97e Add `Truncate` method (#2220)
This PR adds a `Truncate` method which allows to delete all existing rows in a table. The current `Delete` implementation enforces conditions to prevent accidental data deletion.

Co-authored-by: KN4CK3R <admin@oldschoolhack.me>
Co-authored-by: Lunny Xiao <xiaolunwen@gmail.com>
Reviewed-on: https://gitea.com/xorm/xorm/pulls/2220
Reviewed-by: Lunny Xiao <xiaolunwen@gmail.com>
Co-authored-by: KN4CK3R <kn4ck3r@noreply.gitea.io>
Co-committed-by: KN4CK3R <kn4ck3r@noreply.gitea.io>
2023-02-20 07:17:35 +08:00
datbeohbbh 52855dae32 update go version to v1.17 in .drone.yml (#2219)
issue: #2218
Co-authored-by: datbeohbbh <phongtomfapp@gmail.com>
Reviewed-on: https://gitea.com/xorm/xorm/pulls/2219
Reviewed-by: Lunny Xiao <xiaolunwen@gmail.com>
Co-authored-by: datbeohbbh <datbeohbbh@noreply.gitea.io>
Co-committed-by: datbeohbbh <datbeohbbh@noreply.gitea.io>
2023-02-12 11:16:53 +08:00
jamlacey 0c9963c637 Add support for go-ora driver (#2215)
Co-authored-by: James Lacey <jamlacey@gmail.com>
Reviewed-on: https://gitea.com/xorm/xorm/pulls/2215
Reviewed-by: Lunny Xiao <xiaolunwen@gmail.com>
Co-authored-by: jamlacey <jamlacey@noreply.gitea.io>
Co-committed-by: jamlacey <jamlacey@noreply.gitea.io>
2023-02-04 21:24:29 +08:00
datbeohbbh 7dc2a18876 Bug fix: `Rows` must be closed after used (#2214)
Issue: [#2213](https://gitea.com/xorm/xorm/issues/2213#issue-132724)
Co-authored-by: datbeohbbh <phongtomfapp@gmail.com>
Reviewed-on: https://gitea.com/xorm/xorm/pulls/2214
Reviewed-by: Lunny Xiao <xiaolunwen@gmail.com>
Co-authored-by: datbeohbbh <datbeohbbh@noreply.gitea.io>
Co-committed-by: datbeohbbh <datbeohbbh@noreply.gitea.io>
2023-02-03 17:24:16 +08:00
tamalsaha 5fafa00043 fix: Correctly parse jsonb column tag (#2206)
Signed-off-by: Tamal Saha <tamal@appscode.com>
Co-authored-by: Tamal Saha <tamal@appscode.com>
Reviewed-on: https://gitea.com/xorm/xorm/pulls/2206
Reviewed-by: Lunny Xiao <xiaolunwen@gmail.com>
Co-authored-by: tamalsaha <tamalsaha@noreply.gitea.io>
Co-committed-by: tamalsaha <tamalsaha@noreply.gitea.io>
2023-01-09 13:19:29 +08:00
Dmitry Narizhnykh 71270edfcc add delimiters between COMMENT ON COLUMN... for Postgres (#2156)
Fix bug: CreateTableSQL func generates invalid SQL without ";" delimiters.

Reviewed-on: https://gitea.com/xorm/xorm/pulls/2156
Reviewed-by: Lunny Xiao <xiaolunwen@gmail.com>
Co-authored-by: Dmitry Narizhnykh <slotix@noreply.gitea.io>
Co-committed-by: Dmitry Narizhnykh <slotix@noreply.gitea.io>
2022-12-12 18:35:40 +08:00
Lunny Xiao f1bfc5ce98 join support condition (#2201)
Reviewed-on: https://gitea.com/xorm/xorm/pulls/2201
2022-12-09 23:37:26 +08:00
tylerthail2019 71a5939c65 add disable version check func (#2197)
Co-authored-by: tyler <tylerthail@gmail.com>
Reviewed-on: https://gitea.com/xorm/xorm/pulls/2197
Reviewed-by: Lunny Xiao <xiaolunwen@gmail.com>
Co-authored-by: tylerthail2019 <tylerthail2019@noreply.gitea.io>
Co-committed-by: tylerthail2019 <tylerthail2019@noreply.gitea.io>
2022-11-16 13:22:04 +08:00
stevefan1999 3acabdaf26 Fix Oracle Table creation default value (#2190)
Reviewed-on: https://gitea.com/xorm/xorm/pulls/2190
Reviewed-by: Lunny Xiao <xiaolunwen@gmail.com>
Co-authored-by: stevefan1999 <stevefan1999@noreply.gitea.io>
Co-committed-by: stevefan1999 <stevefan1999@noreply.gitea.io>
2022-10-24 11:29:54 +08:00
Lunny Xiao bd58520020
add changelog for 1.3.2 2022-09-03 15:14:19 +08:00
Andrew Thornton c900ecc87f Prevent Sync failure with non-regular indexes on Postgres (#2174)
When dropping indexes in Postgres if the index is non-regular we
should not attempt to regularise the index name as it is already correct.

Signed-off-by: Andrew Thornton <art27@cantab.net>

Reviewed-on: https://gitea.com/xorm/xorm/pulls/2174
Reviewed-by: Lunny Xiao <xiaolunwen@gmail.com>
Co-authored-by: Andrew Thornton <art27@cantab.net>
Co-committed-by: Andrew Thornton <art27@cantab.net>
2022-09-03 10:12:17 +08:00
Andrew Thornton c3bce55620 Change schemas.Column to use int64 (#2160)
Reviewed-on: https://gitea.com/xorm/xorm/pulls/2160
Reviewed-by: Lunny Xiao <xiaolunwen@gmail.com>
Co-authored-by: Andrew Thornton <art27@cantab.net>
Co-committed-by: Andrew Thornton <art27@cantab.net>
2022-07-14 13:55:24 +08:00
Lunny Xiao f469d88166
Update changelog for 1.3.1 2022-06-03 17:44:49 +08:00
Lunny Xiao c98930f8f2 Fix oid index for postgres (#2154)
Reviewed-on: https://gitea.com/xorm/xorm/pulls/2154
2022-06-03 15:24:24 +08:00
Lunny Xiao f9a6990ecb Refactor orderby and support arguments (#2150)
Reviewed-on: https://gitea.com/xorm/xorm/pulls/2150
2022-05-31 11:00:28 +08:00
Andrew Thornton eeb7fcf22c Add ORDER BY SEQ_IN_INDEX to MySQL GetIndexes to Fix IndexTests (#2152)
Reviewed-on: https://gitea.com/xorm/xorm/pulls/2152
Reviewed-by: Lunny Xiao <xiaolunwen@gmail.com>
Co-authored-by: Andrew Thornton <art27@cantab.net>
Co-committed-by: Andrew Thornton <art27@cantab.net>
2022-05-30 18:36:23 +08:00
Andrew Thornton 60540cbabe Remove const insertSelectPlaceholder and associated dead code (#2151)
`insertSelectPlaceholder` is an unexported const set at true. No code changes this nor can any build environment change it.

Therefore we should remove it and the associated dead code.

Close #2146

Signed-off-by: Andrew Thornton <art27@cantab.net>
Reviewed-on: https://gitea.com/xorm/xorm/pulls/2151
Reviewed-by: Lunny Xiao <xiaolunwen@gmail.com>
Co-authored-by: Andrew Thornton <art27@cantab.net>
Co-committed-by: Andrew Thornton <art27@cantab.net>
2022-05-30 00:29:03 +08:00
Andrew Thornton 26d291bbc3 Add interface to allow structs to provide specific index information (#2137)
The current mechanism for adding information about indices cannot express the ordering
of the columns in the index or add sorting information.

Here we add a new interface TableIndices which a struct would implement to provide
a slice of *schema.Index to provide additional indices to that gleaned from the tags.

Signed-off-by: Andrew Thornton <art27@cantab.net>

Reviewed-on: https://gitea.com/xorm/xorm/pulls/2137
Reviewed-by: Lunny Xiao <xiaolunwen@gmail.com>
Co-authored-by: Andrew Thornton <art27@cantab.net>
Co-committed-by: Andrew Thornton <art27@cantab.net>
2022-04-24 19:34:27 +08:00
Lunny Xiao f7e9fb74ac return a clear error for set TEXT type as compare condition (#2062)
Fix #523

Reviewed-on: https://gitea.com/xorm/xorm/pulls/2062
2022-04-23 17:19:37 +08:00
Lunny Xiao 2c064b6da6 Add test for find date (#2121)
Reviewed-on: https://gitea.com/xorm/xorm/pulls/2121
2022-04-22 14:56:26 +08:00
Pierre-Louis Bonicoli e1d4365667 MySQL/MariaDB: return max length for text columns (#2133)
MySQL/MariaDB: return max length for text columns using `CHARACTER_MAXIMUM_LENGTH`.

 Tests:
 * add an integration test: `TestGetColumnsLength`
 * update `TestSyncTable3` since `TableInfo` isn't able to provide the column size

Co-authored-by: Pierre-Louis Bonicoli <pierre-louis.bonicoli@libregerbil.fr>
Co-authored-by: Lunny Xiao <xiaolunwen@gmail.com>
Reviewed-on: https://gitea.com/xorm/xorm/pulls/2133
Reviewed-by: Lunny Xiao <xiaolunwen@gmail.com>
Co-authored-by: Pierre-Louis Bonicoli <pilou@noreply.gitea.io>
Co-committed-by: Pierre-Louis Bonicoli <pilou@noreply.gitea.io>
2022-04-22 10:48:53 +08:00
Lunny Xiao 8f2596bf64 some improvement (#2136)
Fix #2134 and replace #2135

Reviewed-on: https://gitea.com/xorm/xorm/pulls/2136
2022-04-22 10:16:35 +08:00