Commit Graph

1763 Commits

Author SHA1 Message Date
arturwwl 24a672be3c convert - String2Time accept HH:mm:ss format (#2074)
resolves #2073

Co-authored-by: arturwwl <>
Co-authored-by: Lunny Xiao <xiaolunwen@gmail.com>
Reviewed-on: https://gitea.com/xorm/xorm/pulls/2074
Co-authored-by: arturwwl <arturwwl@noreply.gitea.com>
Co-committed-by: arturwwl <arturwwl@noreply.gitea.com>
2023-07-23 02:34:10 +00:00
Lunny Xiao 3626de1459 mod tidy (#2303)
Reviewed-on: https://gitea.com/xorm/xorm/pulls/2303
2023-07-23 01:31:59 +00:00
datbeohbbh c0d09c0def update go version in `go.mod` (#2224)
Hi! I would like to request an update to the `go.mod` file.

Co-authored-by: datbeohbbh <phongtomfapp@gmail.com>
Co-authored-by: Lunny Xiao <xiaolunwen@gmail.com>
Reviewed-on: https://gitea.com/xorm/xorm/pulls/2224
Co-authored-by: datbeohbbh <datbeohbbh@noreply.gitea.com>
Co-committed-by: datbeohbbh <datbeohbbh@noreply.gitea.com>
2023-07-23 01:30:49 +00:00
Lunny Xiao 6c29ab378e refactor write insert sql (#2302)
Reviewed-on: https://gitea.com/xorm/xorm/pulls/2302
2023-07-22 15:24:19 +00:00
Lunny Xiao 9988dac44d improve write order by (#2301)
Reviewed-on: https://gitea.com/xorm/xorm/pulls/2301
2023-07-22 11:52:38 +00:00
Lunny Xiao 96ed5584e3 move sql geneartion for update to statement (#2300)
Reviewed-on: https://gitea.com/xorm/xorm/pulls/2300
2023-07-21 16:25:58 +00:00
Lunny Xiao dabcb4c0ee Return error if count returned no row (#2298)
Fix #2139

Reviewed-on: https://gitea.com/xorm/xorm/pulls/2298
2023-07-20 14:48:29 +00:00
Lunny Xiao 2df56f033a Some refactors (#2293)
Reviewed-on: https://gitea.com/xorm/xorm/pulls/2293
2023-07-20 14:45:31 +00:00
LinkinStars 9b71cb49cc The loadTableInfo function supports passing the context. (#2297)
## `loadTableInfo` add context parameter

### Main change
```diff
+++ func (engine *Engine) loadTableInfo(ctx context.Context, table *schemas.Table) error
--- func (engine *Engine) loadTableInfo(table *schemas.Table) error
```

### Situation
After #2200, I built custom dialect to control the SQL. I find that everything else is fine, except when the `SYNC` method executes with an exception.
The reason is that the `loadTableInfo` method calls the `GetIndexes` and `GetColumns` methods with the dialect during execution. **The context passed to these two methods are all `engine.defaultContext` not the current session's context.** So, I think the `loadTableInfo` method should add the context parameter to ensure that the correct context is used during execution.

### Review Note
1. dialect's `GetColumns` and `GetIndexes` methods are **only** used here, if the context here is incorrect, then the context parameter is invalid.
2. `loadTableInfo` method is only used in `SYNC` and `DBMetas` methods, `SYNC` should pass the session's context, while `DBMetas` has no problem passing `engine.defaultContext`.

All in all, I think this change should not affect other function.

Reviewed-on: https://gitea.com/xorm/xorm/pulls/2297
Co-authored-by: LinkinStars <linkinstar@foxmail.com>
Co-committed-by: LinkinStars <linkinstar@foxmail.com>
2023-07-14 07:35:35 +00:00
CyJaySong 0b18440d73 fix time parse layout (#2296)
Reviewed-on: https://gitea.com/xorm/xorm/pulls/2296
Co-authored-by: CyJaySong <cyjaysong@gmail.com>
Co-committed-by: CyJaySong <cyjaysong@gmail.com>
2023-07-14 07:34:02 +00:00
Lunny Xiao f33221df74 Fix question mark replacement on postgres (#2202)
Fix #2112

Reviewed-on: https://gitea.com/xorm/xorm/pulls/2202
2023-07-12 13:22:32 +00:00
takumin 722f1cc141 Generate coverage html (#2170)
Co-authored-by: Lunny Xiao <xiaolunwen@gmail.com>
Reviewed-on: https://gitea.com/xorm/xorm/pulls/2170
Co-authored-by: takumin <takumiiinn@gmail.com>
Co-committed-by: takumin <takumiiinn@gmail.com>
2023-07-12 13:01:34 +00:00
Lunny Xiao 3b53a5f847 upgrade dependencies (#2223)
Fix #2209

Reviewed-on: https://gitea.com/xorm/xorm/pulls/2223
2023-07-12 11:06:39 +00:00
洪坤安 73eee961cc Performance Optimization: reduce slice2Bean calling times of strings.ToLower and map creation times (#2255)
1. 优化性能:减少func (session *Session) slice2Bean方法中的strings.ToLower调用次数以及减少map创建和访问次数(见工单 https://gitea.com/xorm/xorm/issues/2243)

2. 新增SetDefaultJSONHandler方法,用于用户自行设置DefaultJSONHandler(见工单 https://gitea.com/xorm/xorm/issues/2129)

优化前:
加载耗时=16.079s,总记录数=4028940
加载耗时=15.775s,总记录数=4028940
加载耗时=15.946s,总记录数=4028940

优化后:
加载耗时=10.863s,总记录数=4028940
加载耗时=11.257s,总记录数=4028940
加载耗时=11.155s,总记录数=4028940

Co-authored-by: Lunny Xiao <xiaolunwen@gmail.com>
Reviewed-on: https://gitea.com/xorm/xorm/pulls/2255
Co-authored-by: 洪坤安 <wuzili1234@163.com>
Co-committed-by: 洪坤安 <wuzili1234@163.com>
2023-07-12 09:53:19 +00:00
martinvigg 55d9407390 mysql: add CHAIN, RANK to reserved word list (#2282)
Reference: https://dev.mysql.com/doc/refman/8.0/en/keywords.html
Co-authored-by: Martin Viggiano <martin.viggiano@stonebranch.com>
Co-authored-by: Lunny Xiao <xiaolunwen@gmail.com>
Reviewed-on: https://gitea.com/xorm/xorm/pulls/2282
Co-authored-by: martinvigg <martinvigg@noreply.gitea.com>
Co-committed-by: martinvigg <martinvigg@noreply.gitea.com>
2023-07-12 08:52:23 +00:00
Lunny Xiao f1f5e7cd1a Some refactor (#2292)
replace #2285

Reviewed-on: https://gitea.com/xorm/xorm/pulls/2292
2023-07-12 07:53:25 +00:00
Lunny Xiao 79a8bc804b Fix join problem (#2291)
Fix #2284

Reviewed-on: https://gitea.com/xorm/xorm/pulls/2291
2023-07-12 02:01:56 +00:00
brookechen 486c344ba3 In SQLite3, Sync doesn't support Modify Column:Error: near MODIFY: syntax error (#2267)
rebase

Co-authored-by: brookechen <brookechen@noreply.gitea.io>
Co-authored-by: brookechen <brookechen@tencent.com>
Co-authored-by: Lunny Xiao <xiaolunwen@gmail.com>
Reviewed-on: https://gitea.com/xorm/xorm/pulls/2267
Co-authored-by: brookechen <brookechen@noreply.gitea.com>
Co-committed-by: brookechen <brookechen@noreply.gitea.com>
2023-07-11 17:10:36 +00:00
flyingpigge 52b01ce67f chore: ignore unnecessary char type cast in GetColumns (#2278)
PostgreSQL有两种char类型:`character`和`"char"`: https://www.postgresql.org/docs/current/datatype-character.html.

`pg_class`里的`relkind`是`"char"`类型,所以`GetColumns`这里应该转成`::"char"`或者和本PR里请求的一样去掉转换。这样对于PostgreSQL以及其他兼容PostgreSQL的数据库容错性更好,在做比较时它们通常都会被隐式转换。
```sql
postgres=# select pg_typeof(relkind), pg_typeof('a'::char), pg_typeof('a'::"char") from pg_class limit 1;
 pg_typeof | pg_typeof | pg_typeof
-----------+-----------+-----------
 "char"    | character | "char"
(1 row)
```

Co-authored-by: August <pin@singularity-data.com>
Co-authored-by: Lunny Xiao <xiaolunwen@gmail.com>
Reviewed-on: https://gitea.com/xorm/xorm/pulls/2278
Co-authored-by: flyingpigge <flyingpigge@noreply.gitea.com>
Co-committed-by: flyingpigge <flyingpigge@noreply.gitea.com>
2023-07-03 09:15:45 +00:00
Lunny Xiao d29fe49933 Mysql support a new tag Collate (#2283)
Fix #237
Fix #2179

Reviewed-on: https://gitea.com/xorm/xorm/pulls/2283
2023-07-01 03:40:09 +00:00
Lunny Xiao 18f8e7a86c Default don't log warn for database extra columns when syncing (#2280)
Reviewed-on: https://gitea.com/xorm/xorm/pulls/2280
2023-06-23 15:00:31 +00:00
Lunny Xiao 068de8c0f8 Don't warn when database have extra columns which are not in struct (#2279)
Reviewed-on: https://gitea.com/xorm/xorm/pulls/2279
2023-06-23 08:48:37 +00:00
ccbhj 838a0d9bca ccbhj/fix_mysql_blob_shared_bytes (#2274)
Copy the sql.RawBytes when converting to []byte.
Fix issue https://gitea.com/xorm/xorm/issues/2273

Co-authored-by: Bingjia Chen <bingjia.chen@seamoney.com>
Reviewed-on: https://gitea.com/xorm/xorm/pulls/2274
Reviewed-by: Lunny Xiao <xiaolunwen@gmail.com>
Co-authored-by: ccbhj <ccbhj@noreply.gitea.com>
Co-committed-by: ccbhj <ccbhj@noreply.gitea.com>
2023-06-12 09:18:13 +00:00
Lunny Xiao caa8a029c6 some optimzation (#2272)
Reviewed-on: https://gitea.com/xorm/xorm/pulls/2272
2023-06-02 14:16:30 +00:00
LinkinStars cb851a2f95 Filter support passing context (#2200) (#2270)
```diff
// Filter is an interface to filter SQL
type Filter interface {
---	Do(sql string) string
+++	Do(ctx context.Context, sql string) string
}
```

### Adds a `Context` parameter to the `Do` method of the Filter interface.

Developers can rewrite SQL through the `Filter` `Do` method and **need to get the necessary data from the Context** to assist.

For example, get user information through `Context`, so that different users can use different tables. Another example is to get the flags through `Context` to add annotations to the SQL, and use the annotations to let the subsequent `DB-Proxy` to achieve read/write separation.

Reviewed-on: https://gitea.com/xorm/xorm/pulls/2270
Reviewed-by: Lunny Xiao <xiaolunwen@gmail.com>
Co-authored-by: LinkinStars <linkinstar@foxmail.com>
Co-committed-by: LinkinStars <linkinstar@foxmail.com>
2023-05-31 01:43:24 +00:00
Zettat123 57f7d69f1b Fix test-cockroach workflow (#2269)
The image has already specified the entrypoint so we don't need to call `/cockroach/cockroach` in `cmd`.
![image](/attachments/669496c1-5673-4609-abc7-f42846e4d479)

Reviewed-on: https://gitea.com/xorm/xorm/pulls/2269
Co-authored-by: Zettat123 <zettat123@gmail.com>
Co-committed-by: Zettat123 <zettat123@gmail.com>
2023-05-29 13:01:43 +00:00
Lunny Xiao 04d36cfa81 Use actions instead of drone (#2258)
Reviewed-on: https://gitea.com/xorm/xorm/pulls/2258
2023-05-25 18:15:09 +08:00
brookechen 190384b4cd AutoIncrement列带ID插入数据时没有Commit (#2264)
业务场景中,需要预留(1 ~ 100)的ID给系统规则使用。所以会先使用插入将AutoIncrement列的id偏移到一个特定的值(如:100),然后“带ID调用Insert插入系统规则”。 当带ID插入时,由于没有commit,会被rollback掉。

Reviewed-on: https://gitea.com/xorm/xorm/pulls/2264
Reviewed-by: Lunny Xiao <xiaolunwen@gmail.com>
Co-authored-by: brookechen <brookechen@noreply.gitea.io>
Co-committed-by: brookechen <brookechen@noreply.gitea.io>
2023-05-17 15:20:40 +08:00
Martin Viggiano e6907e9a62 oracle: Fix quotes on DropTableSQL function, add IsSequenceExist function. (#2265)
- Fixed quotes on drop table SQL statement.
- Implemented IsSequenceExists function for oracle dialect.

Reviewed-on: https://gitea.com/xorm/xorm/pulls/2265
Reviewed-by: Lunny Xiao <xiaolunwen@gmail.com>
Co-authored-by: Martin Viggiano <martinvigg@gmail.com>
Co-committed-by: Martin Viggiano <martinvigg@gmail.com>
2023-05-17 09:04:00 +08:00
Abei1uo 23be940bad Fix session insert interface slice commit panic(#2259) (#2260)
Fix session insert interface slice commit panic(#2259)
-- TestInsertMulti2InterfaceTransaction is the test case

Reviewed-on: https://gitea.com/xorm/xorm/pulls/2260
Co-authored-by: Abei1uo <2676567028@qq.com>
Co-committed-by: Abei1uo <2676567028@qq.com>
2023-05-12 14:58:38 +08:00
Alberto Garcia 94fcec7f65 parse timestamp with milliseconds in tz format (#2246)
This pr fixes a bug caused when a timestmap in TZ format with milliseconds (`2023-04-05T15:50:48.256816Z` or `2023-04-05T15:50:48.256816+08:00`) is being parsed.

- The bug is happens in the function `String2Time` in `convert/time.go`
- if the timestamp contains milliseconds a layout with the structure `2006-01-02 15:04:05.` is used to parse it
- This layout contains an space between the date and the hour, if the timestamp returned by the db contains a `T` the parser fails and an error is returned

This pr adds a check for the `T` and milliseconds in the timestamp, if both conditions are present the date is parsed using the RFC3339 layout

Added test cases to check timestamps in this format

Solves [#2244](https://gitea.com/xorm/xorm/issues/2244)

Reviewed-on: https://gitea.com/xorm/xorm/pulls/2246
Reviewed-by: Lunny Xiao <xiaolunwen@gmail.com>
Co-authored-by: Alberto Garcia <agarcia@circutor.com>
Co-committed-by: Alberto Garcia <agarcia@circutor.com>
2023-04-06 21:47:59 +08:00
datbeohbbh d485abba57 add `(*Engine) SetConnMaxIdleTime` (#2229)
issue: https://gitea.com/xorm/xorm/issues/2228
Co-authored-by: datbeohbbh <phongtomfapp@gmail.com>
Reviewed-on: https://gitea.com/xorm/xorm/pulls/2229
Reviewed-by: Lunny Xiao <xiaolunwen@gmail.com>
Co-authored-by: datbeohbbh <datbeohbbh@noreply.gitea.io>
Co-committed-by: datbeohbbh <datbeohbbh@noreply.gitea.io>
2023-03-06 18:55:33 +08:00
fanshengshuai 914f2db9ea mysql字段为UNSIGNED时,会导致字段类型无法识别,返回RawBytes,JSON 字段映射为String (#2225)
如题

Reviewed-on: https://gitea.com/xorm/xorm/pulls/2225
Reviewed-by: Lunny Xiao <xiaolunwen@gmail.com>
Co-authored-by: fanshengshuai <fanshengshuai@noreply.gitea.io>
Co-committed-by: fanshengshuai <fanshengshuai@noreply.gitea.io>
2023-02-28 23:42:42 +08:00
KN4CK3R 056cecc97e Add `Truncate` method (#2220)
This PR adds a `Truncate` method which allows to delete all existing rows in a table. The current `Delete` implementation enforces conditions to prevent accidental data deletion.

Co-authored-by: KN4CK3R <admin@oldschoolhack.me>
Co-authored-by: Lunny Xiao <xiaolunwen@gmail.com>
Reviewed-on: https://gitea.com/xorm/xorm/pulls/2220
Reviewed-by: Lunny Xiao <xiaolunwen@gmail.com>
Co-authored-by: KN4CK3R <kn4ck3r@noreply.gitea.io>
Co-committed-by: KN4CK3R <kn4ck3r@noreply.gitea.io>
2023-02-20 07:17:35 +08:00
datbeohbbh 52855dae32 update go version to v1.17 in .drone.yml (#2219)
issue: #2218
Co-authored-by: datbeohbbh <phongtomfapp@gmail.com>
Reviewed-on: https://gitea.com/xorm/xorm/pulls/2219
Reviewed-by: Lunny Xiao <xiaolunwen@gmail.com>
Co-authored-by: datbeohbbh <datbeohbbh@noreply.gitea.io>
Co-committed-by: datbeohbbh <datbeohbbh@noreply.gitea.io>
2023-02-12 11:16:53 +08:00
jamlacey 0c9963c637 Add support for go-ora driver (#2215)
Co-authored-by: James Lacey <jamlacey@gmail.com>
Reviewed-on: https://gitea.com/xorm/xorm/pulls/2215
Reviewed-by: Lunny Xiao <xiaolunwen@gmail.com>
Co-authored-by: jamlacey <jamlacey@noreply.gitea.io>
Co-committed-by: jamlacey <jamlacey@noreply.gitea.io>
2023-02-04 21:24:29 +08:00
datbeohbbh 7dc2a18876 Bug fix: `Rows` must be closed after used (#2214)
Issue: [#2213](https://gitea.com/xorm/xorm/issues/2213#issue-132724)
Co-authored-by: datbeohbbh <phongtomfapp@gmail.com>
Reviewed-on: https://gitea.com/xorm/xorm/pulls/2214
Reviewed-by: Lunny Xiao <xiaolunwen@gmail.com>
Co-authored-by: datbeohbbh <datbeohbbh@noreply.gitea.io>
Co-committed-by: datbeohbbh <datbeohbbh@noreply.gitea.io>
2023-02-03 17:24:16 +08:00
tamalsaha 5fafa00043 fix: Correctly parse jsonb column tag (#2206)
Signed-off-by: Tamal Saha <tamal@appscode.com>
Co-authored-by: Tamal Saha <tamal@appscode.com>
Reviewed-on: https://gitea.com/xorm/xorm/pulls/2206
Reviewed-by: Lunny Xiao <xiaolunwen@gmail.com>
Co-authored-by: tamalsaha <tamalsaha@noreply.gitea.io>
Co-committed-by: tamalsaha <tamalsaha@noreply.gitea.io>
2023-01-09 13:19:29 +08:00
Dmitry Narizhnykh 71270edfcc add delimiters between COMMENT ON COLUMN... for Postgres (#2156)
Fix bug: CreateTableSQL func generates invalid SQL without ";" delimiters.

Reviewed-on: https://gitea.com/xorm/xorm/pulls/2156
Reviewed-by: Lunny Xiao <xiaolunwen@gmail.com>
Co-authored-by: Dmitry Narizhnykh <slotix@noreply.gitea.io>
Co-committed-by: Dmitry Narizhnykh <slotix@noreply.gitea.io>
2022-12-12 18:35:40 +08:00
Lunny Xiao f1bfc5ce98 join support condition (#2201)
Reviewed-on: https://gitea.com/xorm/xorm/pulls/2201
2022-12-09 23:37:26 +08:00
tylerthail2019 71a5939c65 add disable version check func (#2197)
Co-authored-by: tyler <tylerthail@gmail.com>
Reviewed-on: https://gitea.com/xorm/xorm/pulls/2197
Reviewed-by: Lunny Xiao <xiaolunwen@gmail.com>
Co-authored-by: tylerthail2019 <tylerthail2019@noreply.gitea.io>
Co-committed-by: tylerthail2019 <tylerthail2019@noreply.gitea.io>
2022-11-16 13:22:04 +08:00
stevefan1999 3acabdaf26 Fix Oracle Table creation default value (#2190)
Reviewed-on: https://gitea.com/xorm/xorm/pulls/2190
Reviewed-by: Lunny Xiao <xiaolunwen@gmail.com>
Co-authored-by: stevefan1999 <stevefan1999@noreply.gitea.io>
Co-committed-by: stevefan1999 <stevefan1999@noreply.gitea.io>
2022-10-24 11:29:54 +08:00
Lunny Xiao bd58520020
add changelog for 1.3.2 2022-09-03 15:14:19 +08:00
Andrew Thornton c900ecc87f Prevent Sync failure with non-regular indexes on Postgres (#2174)
When dropping indexes in Postgres if the index is non-regular we
should not attempt to regularise the index name as it is already correct.

Signed-off-by: Andrew Thornton <art27@cantab.net>

Reviewed-on: https://gitea.com/xorm/xorm/pulls/2174
Reviewed-by: Lunny Xiao <xiaolunwen@gmail.com>
Co-authored-by: Andrew Thornton <art27@cantab.net>
Co-committed-by: Andrew Thornton <art27@cantab.net>
2022-09-03 10:12:17 +08:00
Andrew Thornton c3bce55620 Change schemas.Column to use int64 (#2160)
Reviewed-on: https://gitea.com/xorm/xorm/pulls/2160
Reviewed-by: Lunny Xiao <xiaolunwen@gmail.com>
Co-authored-by: Andrew Thornton <art27@cantab.net>
Co-committed-by: Andrew Thornton <art27@cantab.net>
2022-07-14 13:55:24 +08:00
Lunny Xiao f469d88166
Update changelog for 1.3.1 2022-06-03 17:44:49 +08:00
Lunny Xiao c98930f8f2 Fix oid index for postgres (#2154)
Reviewed-on: https://gitea.com/xorm/xorm/pulls/2154
2022-06-03 15:24:24 +08:00
Lunny Xiao f9a6990ecb Refactor orderby and support arguments (#2150)
Reviewed-on: https://gitea.com/xorm/xorm/pulls/2150
2022-05-31 11:00:28 +08:00
Andrew Thornton eeb7fcf22c Add ORDER BY SEQ_IN_INDEX to MySQL GetIndexes to Fix IndexTests (#2152)
Reviewed-on: https://gitea.com/xorm/xorm/pulls/2152
Reviewed-by: Lunny Xiao <xiaolunwen@gmail.com>
Co-authored-by: Andrew Thornton <art27@cantab.net>
Co-committed-by: Andrew Thornton <art27@cantab.net>
2022-05-30 18:36:23 +08:00
Andrew Thornton 60540cbabe Remove const insertSelectPlaceholder and associated dead code (#2151)
`insertSelectPlaceholder` is an unexported const set at true. No code changes this nor can any build environment change it.

Therefore we should remove it and the associated dead code.

Close #2146

Signed-off-by: Andrew Thornton <art27@cantab.net>
Reviewed-on: https://gitea.com/xorm/xorm/pulls/2151
Reviewed-by: Lunny Xiao <xiaolunwen@gmail.com>
Co-authored-by: Andrew Thornton <art27@cantab.net>
Co-committed-by: Andrew Thornton <art27@cantab.net>
2022-05-30 00:29:03 +08:00