diff --git a/CONTRIBUTING.md b/CONTRIBUTING.md index 4b806704..90f630ea 100644 --- a/CONTRIBUTING.md +++ b/CONTRIBUTING.md @@ -9,6 +9,15 @@ conventions when submitting patches. * [fork a repo](https://help.github.com/articles/fork-a-repo) * [creating a pull request ](https://help.github.com/articles/creating-a-pull-request) +### Sign your codes with comments +``` +// !! your comments + +e.g., + +// !lunny! this is comments made by lunny +``` + ### Patch review Help review existing open [pull requests](https://help.github.com/articles/using-pull-requests) by commenting on the code or @@ -17,7 +26,7 @@ proposed functionality. ### Bug reports We appreciate any bug reports, but especially ones with self-contained -(doesn't depend on code outside of pq), minimal (can't be simplified +(doesn't depend on code outside of xorm), minimal (can't be simplified further) test cases. It's especially helpful if you can submit a pull request with just the failing test case (you'll probably want to pattern it after the tests in diff --git a/README.md b/README.md index 9c9ecfa7..730f890a 100644 --- a/README.md +++ b/README.md @@ -18,7 +18,7 @@ Xorm is a simple and powerful ORM for Go. * Query Cache speed up -* Database Reverse support, See [Xorm Tool README](https://github.com/go-xorm/xorm/blob/master/xorm/README.md) +* Database Reverse support, See [Xorm Tool README](https://github.com/go-xorm/cmd/blob/master/README.md) * Simple cascade loading support @@ -96,7 +96,7 @@ Or * [GoWalker](http://gowalker.org/github.com/go-xorm/xorm) -* [Quick Start](https://github.com/go-xorm/xorm/blob/master/docs/QuickStartEn.md) +* [Quick Start](https://github.com/go-xorm/xorm/blob/master/docs/QuickStart.md) # Cases @@ -123,13 +123,10 @@ Or Please visit [Xorm on Google Groups](https://groups.google.com/forum/#!forum/xorm) -# Contributors +# Contributing If you want to pull request, please see [CONTRIBUTING](https://github.com/go-xorm/xorm/blob/master/CONTRIBUTING.md) -* [Lunny](https://github.com/lunny) -* [Nashtsai](https://github.com/nashtsai) - # LICENSE BSD License diff --git a/README_CN.md b/README_CN.md index c8f3f180..f3974aa3 100644 --- a/README_CN.md +++ b/README_CN.md @@ -94,7 +94,7 @@ xorm是一个简单而强大的Go语言ORM库. 通过它可以使数据库操作 ## 文档 -* [快速开始](https://github.com/go-xorm/xorm/blob/master/docs/QuickStart.md) +* [快速开始](https://github.com/go-xorm/xorm/blob/master/docs/QuickStartCN.md) * [GoWalker代码文档](http://gowalker.org/github.com/go-xorm/xorm) @@ -124,13 +124,10 @@ xorm是一个简单而强大的Go语言ORM库. 通过它可以使数据库操作 请加入QQ群：280360085 进行讨论。 -# 贡献者 +## 贡献 如果您也想为Xorm贡献您的力量，请查看 [CONTRIBUTING](https://github.com/go-xorm/xorm/blob/master/CONTRIBUTING.md) -* [Lunny](https://github.com/lunny) -* [Nashtsai](https://github.com/nashtsai) - ## LICENSE BSD License diff --git a/docs/QuickStart.md b/docs/QuickStart.md index f240806d..434e3f2d 100644 --- a/docs/QuickStart.md +++ b/docs/QuickStart.md @@ -58,7 +58,7 @@ engine, err = xorm.NewEngine("sqlite3", "./test.db") defer engine.Close() ``` -Generally, you can only create one engine. Engine supports run on go rutines. +You can create many engines for different databases.Generally, you just need create only one engine. Engine supports run on go routines. xorm supports four drivers now: @@ -128,23 +128,25 @@ engine.SetColumnMapper(SnakeMapper{}) ### 2.2.Prefix mapping, Suffix Mapping and Cache Mapping -* 通过`engine.NewPrefixMapper(SnakeMapper{}, "prefix")`可以在SnakeMapper的基础上在命名中添加统一的前缀，当然也可以把SnakeMapper{}换成SameMapper或者你自定义的Mapper。 -* 通过`engine.NewSufffixMapper(SnakeMapper{}, "suffix")`可以在SnakeMapper的基础上在命名中添加统一的后缀，当然也可以把SnakeMapper{}换成SameMapper或者你自定义的Mapper。 -* 通过`eneing.NewCacheMapper(SnakeMapper{})`可以起到在内存中缓存曾经映射过的命名映射。 +* `engine.NewPrefixMapper(SnakeMapper{}, "prefix")` can add prefix string when naming based on SnakeMapper or SameMapper, or you custom Mapper. +* `engine.NewPrefixMapper(SnakeMapper{}, "suffix")` can add suffix string when naming based on SnakeMapper or SameMapper, or you custom Mapper. +* `engine.NewCacheMapper(SnakeMapper{})` add naming Mapper for memory cache. -当然，如果你使用了别的命名规则映射方案，也可以自己实现一个IMapper。 +Of course, you can implement IMapper to make custom naming strategy. ### 2.3.Tag mapping -如果所有的命名都是按照IMapper的映射来操作的，那当然是最理想的。但是如果碰到某个表名或者某个字段名跟映射规则不匹配时，我们就需要别的机制来改变。 +It's idealized of using IMapper for all naming. But if table or column is not in rule, we need new method to archive. -通过`engine.Table()`方法可以改变struct对应的数据库表的名称，通过sturct中field对应的Tag中使用`xorm:"'table_name'"`可以使该field对应的Column名称为指定名称。这里使用两个单引号将Column名称括起来是为了防止名称冲突，因为我们在Tag中还可以对这个Column进行更多的定义。如果名称不冲突的情况，单引号也可以不使用。 +* If struct or pointer of struct has `TableName() string` method, the return value will be the struct's table name. + +* `engine.Table()` can change the database table name for struct. The struct tag `xorm:"'table_name'"` can set column name for struct field. Use a pair of single quotes to prevent confusion for column's definition in struct tag. If not in confusion, ignore single quotes. -### 2.4.Column defenition +### 2.4.Column definition -我们在field对应的Tag中对Column的一些属性进行定义，定义的方法基本和我们写SQL定义表结构类似，比如： +Struct tag defines something for column as basic sql concepts, such as : ``` type User struct { @@ -153,9 +155,9 @@ type User struct { } ``` -For different DBMS, data types对于不同的数据库系统，数据类型其实是有些差异的。因此xorm中对数据类型有自己的定义，基本的原则是尽量兼容各种数据库的字段类型，具体的字段对应关系可以查看[字段类型对应表](https://github.com/go-xorm/xorm/blob/master/docs/COLUMNTYPE.md)。 +Data types are different in different DBMS. So xorm makes own data types definition to keep compatible. Details is in document [Column Types](https://github.com/go-xorm/xorm/blob/master/docs/COLUMNTYPE.md). -具体的映射规则如下，另Tag中的关键字均不区分大小写，字段名区分大小写： +The following table is field mapping rules, the keyword is not case sensitive except column name： @@ -165,7 +167,7 @@ For different DBMS, data types对于不同的数据库系统，数据类型其 - + @@ -174,22 +176,22 @@ For different DBMS, data types对于不同的数据库系统，数据类型其 - + - + - + - + - + @@ -205,16 +207,17 @@ For different DBMS, data types对于不同的数据库系统，数据类型其
pkIf column is Primary Key
当前支持30多种字段类型，详情参见 [字段类型](https://github.com/go-xorm/xorm/blob/master/docs/COLUMNTYPE.md)字段类型support over 30 kinds of column types, details in [Column Types](https://github.com/go-xorm/xorm/blob/master/docs/COLUMNTYPE.md)column type
autoincrIf autoincrement column[not ]null | notnullif column could be blank
unique/unique(uniquename)是否是唯一，如不加括号则该字段不允许重复；如加上括号，则括号中为联合唯一索引的名字，此时如果有另外一个或多个字段和本unique的uniquename相同，则这些uniquename相同的字段组成联合唯一索引unique/unique(uniquename)column is Unique index; if add (uniquename), the column is used for combined unique index with the field that defining same uniquename.
index/index(indexname)是否是索引，如不加括号则该字段自身为索引，如加上括号，则括号中为联合索引的名字，此时如果有另外一个或多个字段和本index的indexname相同，则这些indexname相同的字段组成联合索引index/index(indexname)column is index. if add (indexname), the column is used for combined index with the field that defining same indexname.
extends应用于一个匿名结构体之上，表示此匿名结构体的成员也映射到数据库中extendsuse for anonymous field, map the struct in anonymous field to database
-This field will not be mapping
->这个Field将只写入到数据库而不从数据库读取->only write into database
<-这个Field将只从数据库读取，而不写入到数据库<-only read from database
createdThis field will be filled in current time on insert
-另外有如下几条自动映射的规则： +Some default mapping rules： -- 1.如果field名称为`Id`而且类型为`int64`的话，会被xorm视为主键，并且拥有自增属性。如果想用`Id`以外的名字做为主键名，可以在对应的Tag上加上`xorm:"pk"`来定义主键。 +- 1. If field is name of `Id` and type of `int64`, xorm makes it as auto increment primary key. If another field, use struct tag `xorm:"pk"`. -- 2.string类型默认映射为varchar(255)，如果需要不同的定义，可以在tag中自定义 +- 2. String is corresponding to varchar(255). -- 3.支持`type MyString string`等自定义的field，支持Slice, Map等field成员，这些成员默认存储为Text类型，并且默认将使用Json格式来序列化和反序列化。也支持数据库字段类型为Blob类型，如果是Blob类型，则先使用Json格式序列化再转成[]byte格式。当然[]byte或者[]uint8默认为Blob类型并且都以二进制方式存储。 +- 3. Support custom type as `type MyString string`，slice, map as field type. They are saving as Text column type and json-encode string. Support Blob column type with field type []byte or []uint8. -- 4.实现了Conversion接口的类型或者结构体，将根据接口的转换方式在类型和数据库记录之间进行相互转换。 -```Go +- 4. You can implement Conversion interface to define your custom mapping rule between field and database data. + +``` type Conversion interface { FromDB([]byte) error ToDB() ([]byte, error) @@ -222,55 +225,55 @@ type Conversion interface { ``` -## 3.表结构操作 +## 3. database meta information -xorm提供了一些动态获取和修改表结构的方法。对于一般的应用，很少动态修改表结构，则只需调用Sync()同步下表结构即可。 +xorm provides methods to getting and setting table schema. For less schema changing production, `engine.Sync()` is enough. ## 3.1 retrieve database meta info * DBMetas() -xorm支持获取表结构信息，通过调用`engine.DBMetas()`可以获取到所有的表的信息 +`engine.DBMetas()` returns all tables schema information. ## 3.2.directly table operation * CreateTables() -创建表使用`engine.CreateTables()`，参数为一个或多个空的对应Struct的指针。同时可用的方法有Charset()和StoreEngine()，如果对应的数据库支持，这两个方法可以在创建表时指定表的字符编码和使用的引擎。当前仅支持Mysql数据库。 +`engine.CreateTables(struct)` creates table with struct or struct pointer. +`engine.Charset()` and `engine.StoreEngine()` can change charset or storage engine for **mysql** database. * IsTableEmpty() -判断表是否为空，参数和CreateTables相同 +check table is empty or not. * IsTableExist() -判断表是否存在 +check table is existed or not. * DropTables() -删除表使用`engine.DropTables()`，参数为一个或多个空的对应Struct的指针或者表的名字。如果为string传入，则只删除对应的表，如果传入的为Struct，则删除表的同时还会删除对应的索引。 +`engine.DropTables(struct)` drops table and indexes with struct or struct pointer. `engine.DropTables(string)` only drops table except indexes. ## 3.3.create indexes and uniques * CreateIndexes -根据struct中的tag来创建索引 +create indexes with struct. * CreateUniques -根据struct中的tag来创建唯一索引 +create unique indexes with struct. -## 3.4.同步数据库结构 +## 3.4.Synchronize database schema -同步能够部分智能的根据结构体的变动检测表结构的变动，并自动同步。目前能够实现： -1) 自动检测和创建表，这个检测是根据表的名字 -2）自动检测和新增表中的字段，这个检测是根据字段名 -3）自动检测和创建索引和唯一索引，这个检测是根据一个或多个字段名，而不根据索引名称 +xorm watches tables and indexes and sync schema: +1) use table name to create or drop table +2) use column name to alter column +3) use the indexes definition in struct field tag to create or drop indexes. -调用方法如下： ```Go err := engine.Sync(new(User)) ``` -## 4.插入数据 +## 4.Insert data Inserting records use Insert method. @@ -335,44 +338,43 @@ affected, err := engine.Insert(user, &questions) Notice: If you want to use transaction on inserting, you should use session.Begin() before calling Insert. -## 5.Query and count - -所有的查询条件不区分调用顺序，但必须在调用Get，Find，Count这三个函数之前调用。同时需要注意的一点是，在调用的参数中，所有的字符字段名均为映射后的数据库的字段名，而不是field的名字。 +## 5. Chainable APIs -### 5.1.查询条件方法 +### 5.1. Chainable APIs for Queries, Execusions and Aggregations +Queries and Aggregations is basically formed by using `Get`, `Find`, `Count` methods, with conjunction of following chainable APIs to form conditions, grouping and ordering: 查询和统计主要使用`Get`, `Find`, `Count`三个方法。在进行查询时可以使用多个方法来形成查询条件，条件函数如下： -* Id(int64) -传入一个PK字段的值，作为查询条件 +* Id([]interface{}) +Primary Key lookup * Where(string, …interface{}) -和Where语句中的条件基本相同，作为条件 +As SQL conditional WHERE clause * And(string, …interface{}) -和Where函数中的条件基本相同，作为条件 +Conditional AND * Or(string, …interface{}) -和Where函数中的条件基本相同，作为条件 +Conditional OR * Sql(string, …interface{}) -执行指定的Sql语句，并把结果映射到结构体 +Custom SQL query * Asc(…string) -指定字段名正序排序 +Ascending ordering on 1 or more fields * Desc(…string) -指定字段名逆序排序 +Descending ordering on 1 or more fields * OrderBy(string) -按照指定的顺序进行排序 +As SQL ORDER BY * In(string, …interface{}) -某字段在一些值中 +As SQL Conditional IN * Cols(…string) -只查询或更新某些指定的字段，默认是查询所有映射的字段或者根据Update的第一个参数来判断更新的字段。例如： +Explicity specify query or update columns. e.g.,: ```Go engine.Cols("age", "name").Find(&users) // SELECT age, name FROM user @@ -380,55 +382,59 @@ engine.Cols("age", "name").Update(&user) // UPDATE user SET age=? AND name=? ``` -其中的参数"age", "name"也可以写成"age, name"，两种写法均可 - * Omit(...string) -和cols相反，此函数指定排除某些指定的字段。注意：此方法和Cols方法不可同时使用 +Inverse function to Cols, to exclude specify query or update columns. Warning: Don't use with Cols() ```Go -engine.Cols("age").Update(&user) +engine.Omit("age").Update(&user) // UPDATE user SET name = ? AND department = ? ``` * Distinct(…string) -按照参数中指定的字段归类结果 +As SQL DISTINCT ```Go engine.Distinct("age", "department").Find(&users) // SELECT DISTINCT age, department FROM user ``` -注意：当开启了缓存时，此方法的调用将在当前查询中禁用缓存。因为缓存系统当前依赖Id，而此时无法获得Id +Caution: this method will not lookup from caching store + * Table(nameOrStructPtr interface{}) -传入表名称或者结构体指针，如果传入的是结构体指针，则按照IMapper的规则提取出表名 +Specify table name, or if struct pointer is passed into the name is extract from struct type name by IMapper conversion policy * Limit(int, …int) -限制获取的数目，第一个参数为条数，第二个参数为可选，表示开始位置 +As SQL LIMIT with optional second param for OFFSET * Top(int) -相当于Limit(int, 0) +As SQL LIMIT -* Join(string,string,string) -第一个参数为连接类型，当前支持INNER, LEFT OUTER, CROSS中的一个值，第二个参数为表名，第三个参数为连接条件 +* Join(type, tableName, criteria string) +As SQL JOIN, support +type: either of these values [INNER, LEFT OUTER, CROSS] are supported now +tableName: joining table name +criteria: join criteria * GroupBy(string) -Groupby的参数字符串 +As SQL GROUP BY * Having(string) -Having的参数字符串 +As SQL HAVING -### 5.2.临时开关方法 +### 5.2. Override default behavior APIs * NoAutoTime() -如果此方法执行，则此次生成的语句中Created和Updated字段将不自动赋值为当前时间 +No auto timestamp for Created and Updated fields for INSERT and UPDATE * NoCache() -如果此方法执行，则此次生成的语句则在非缓存模式下执行 +Disable cache lookup + * UseBool(...string) -当从一个struct来生成查询条件或更新字段时，xorm会判断struct的field是否为0,"",nil，如果为以上则不当做查询条件或者更新内容。因为bool类型只有true和false两种值，因此默认所有bool类型不会作为查询条件或者更新字段。如果可以使用此方法，如果默认不传参数，则所有的bool字段都将会被使用，如果参数不为空，则参数中指定的为字段名，则这些字段对应的bool值将被使用。 +xorm's default behavior is fields with 0, "", nil, false, will not be used during query or update, use this method to explicit specify bool type fields for query or update + * Cascade(bool) -是否自动关联查询field中的数据，如果struct的field也是一个struct并且映射为某个Id，则可以在查询时自动调用Get方法查询出对应的数据。 +Do cascade lookup for associations ### 5.3.Get one record @@ -495,9 +501,9 @@ err := engine.Where("age > ? or name=?)", 30, "xlw").Iterate(new(Userinfo), func ``` -### 5.6.Count方法 +### 5.6.Count method usage -统计数据使用`Count`方法，Count方法的参数为struct的指针并且成为查询条件。 +An ORM pointer struct is required for Count method in order to determine which table to retrieve from. ```Go user := new(User) total, err := engine.Where("id >?", 1).Count(user) @@ -620,7 +626,7 @@ if err != nil { ``` -## 11.缓存 +## 11.Built-in LRU memory cache provider 1. Global Cache Xorm implements cache support. Defaultly, it's disabled. If enable it, use below code. @@ -661,15 +667,15 @@ Cache implement theory below: ## 12.xorm tool -xorm工具提供了xorm命令，能够帮助做很多事情。 +xorm commandl line tool ### 12.1.Reverse command -Please visit [xorm tool](https://github.com/go-xorm/xorm/tree/master/xorm) +Please visit [xorm tool](https://github.com/go-xorm/cmd) ## 13.Examples -请访问[https://github.com/go-xorm/xorm/tree/master/examples](https://github.com/go-xorm/xorm/tree/master/examples) +Please visit [https://github.com/go-xorm/xorm/tree/master/examples](https://github.com/go-xorm/xorm/tree/master/examples) ## 14.Cases diff --git a/docs/QuickStartCn.md b/docs/QuickStartCN.md similarity index 98% rename from docs/QuickStartCn.md rename to docs/QuickStartCN.md index 2b76ca8e..638bc50d 100644 --- a/docs/QuickStartCn.md +++ b/docs/QuickStartCN.md @@ -23,7 +23,7 @@ xorm 快速入门 * [5.6.Count方法](#66) * [5.7.Rows方法](#67) * [6.更新数据](#70) -* [6.1.乐观锁](#71) + * [6.1.乐观锁](#71) * [7.删除数据](#80) * [8.执行SQL查询](#90) * [9.执行SQL命令](#100) @@ -62,7 +62,7 @@ engine, err = xorm.NewEngine("sqlite3", "./test.db") defer engine.Close() ``` -一般如果只针对一个数据库进行操作，只需要创建一个Engine即可。Engine支持在多GoRutine下使用。 +你可以创建一个或多个engine, 不过一般如果操作一个数据库，只需要创建一个Engine即可。Engine支持在多GoRutine下使用。 xorm当前支持五种驱动四个数据库如下： @@ -141,7 +141,7 @@ engine.SetColumnMapper(SnakeMapper{}) 如果所有的命名都是按照IMapper的映射来操作的，那当然是最理想的。但是如果碰到某个表名或者某个字段名跟映射规则不匹配时，我们就需要别的机制来改变。 -* 如果struct拥有`Tablename() string`的成员方法，那么此方法的返回值即是该struct默认对应的数据库表名。 +* 如果struct拥有`TableName() string`的成员方法，那么此方法的返回值即是该struct默认对应的数据库表名。 * 通过`engine.Table()`方法可以改变struct对应的数据库表的名称，通过sturct中field对应的Tag中使用`xorm:"'column_name'"`可以使该field对应的Column名称为指定名称。这里使用两个单引号将Column名称括起来是为了防止名称冲突，因为我们在Tag中还可以对这个Column进行更多的定义。如果名称不冲突的情况，单引号也可以不使用。 @@ -419,7 +419,7 @@ engine.Cols("age", "name").Update(&user) * Omit(...string) 和cols相反，此函数指定排除某些指定的字段。注意：此方法和Cols方法不可同时使用 ```Go -engine.Cols("age").Update(&user) +engine.Omit("age").Update(&user) // UPDATE user SET name = ? AND department = ? ``` @@ -767,7 +767,7 @@ xorm支持两种方式的事件，一种是在Struct中的特定方法来作为 xorm工具提供了xorm命令，能够帮助做很多事情。 ### 13.1.反转命令 -参见 [xorm工具](https://github.com/go-xorm/xorm/tree/master/xorm) +参见 [xorm工具](https://github.com/go-xorm/cmd) ## 14.Examples diff --git a/engine.go b/engine.go index 671c1b87..107a9d69 100644 --- a/engine.go +++ b/engine.go @@ -6,6 +6,7 @@ import ( "database/sql" "errors" "fmt" + "io" "os" "reflect" "strconv" @@ -34,8 +35,7 @@ type Engine struct { ShowErr bool ShowDebug bool ShowWarn bool - //Pool IConnectPool - //Filters []core.Filter + Logger ILogger // io.Writer TZLocation *time.Location } @@ -266,6 +266,80 @@ func (engine *Engine) DBMetas() ([]*core.Table, error) { return tables, nil } +func (engine *Engine) DumpAllToFile(fp string) error { + f, err := os.Create(fp) + if err != nil { + return err + } + defer f.Close() + return engine.DumpAll(f) +} + +func (engine *Engine) DumpAll(w io.Writer) error { + tables, err := engine.DBMetas() + if err != nil { + return err + } + + for _, table := range tables { + _, err = io.WriteString(w, engine.dialect.CreateTableSql(table, "", "", "")+"\n\n") + if err != nil { + return err + } + for _, index := range table.Indexes { + _, err = io.WriteString(w, engine.dialect.CreateIndexSql(table.Name, index)+"\n\n") + if err != nil { + return err + } + } + + rows, err := engine.DB().Query("SELECT * FROM " + engine.Quote(table.Name)) + if err != nil { + return err + } + + cols, err := rows.Columns() + if err != nil { + return err + } + if len(cols) == 0 { + continue + } + for rows.Next() { + dest := make([]interface{}, len(cols)) + err = rows.ScanSlice(&dest) + if err != nil { + return err + } + + _, err = io.WriteString(w, "INSERT INTO "+engine.Quote(table.Name)+" ("+engine.Quote(strings.Join(cols, engine.Quote(", ")))+") VALUES (") + if err != nil { + return err + } + + var temp string + for i, d := range dest { + col := table.GetColumn(cols[i]) + if d == nil { + temp += ", NULL" + } else if col.SQLType.IsText() || col.SQLType.IsTime() { + var v = fmt.Sprintf("%s", d) + temp += ", '" + strings.Replace(v, "'", "''", -1) + "'" + } else if col.SQLType.IsBlob() /*reflect.TypeOf(d).Kind() == reflect.Slice*/ { + temp += fmt.Sprintf(", %s", engine.dialect.FormatBytes(d.([]byte))) + } else { + temp += fmt.Sprintf(", %s", d) + } + } + _, err = io.WriteString(w, temp[2:]+");\n\n") + if err != nil { + return err + } + } + } + return nil +} + // use cascade or not func (engine *Engine) Cascade(trueOrFalse ...bool) *Session { session := engine.NewSession() @@ -456,15 +530,6 @@ func (engine *Engine) autoMap(bean interface{}) *core.Table { return engine.autoMapType(v) } -/*func (engine *Engine) mapType(t reflect.Type) *core.Table { - return mappingTable(t, engine.TableMapper, engine.ColumnMapper, engine.dialect, engine.TagIdentifier) -}*/ - -/* -func mappingTable(t reflect.Type, tableMapper core.IMapper, colMapper core.IMapper, dialect core.Dialect, tagId string) *core.Table { - table := core.NewEmptyTable() - table.Name = tableMapper.Obj2Table(t.Name()) -*/ func addIndex(indexName string, table *core.Table, col *core.Column, indexType int) { if index, ok := table.Indexes[indexName]; ok { index.AddColumn(col.Name) @@ -524,17 +589,39 @@ func (engine *Engine) mapType(v reflect.Value) *core.Table { if tags[0] == "-" { continue } - if (strings.ToUpper(tags[0]) == "EXTENDS") && - (fieldType.Kind() == reflect.Struct) { + if strings.ToUpper(tags[0]) == "EXTENDS" { - //parentTable := mappingTable(fieldType, tableMapper, colMapper, dialect, tagId) - parentTable := engine.mapType(fieldValue) - for _, col := range parentTable.Columns() { - col.FieldName = fmt.Sprintf("%v.%v", fieldType.Name(), col.FieldName) - table.AddColumn(col) + //fieldValue = reflect.Indirect(fieldValue) + //fmt.Println("----", fieldValue.Kind()) + if fieldValue.Kind() == reflect.Struct { + //parentTable := mappingTable(fieldType, tableMapper, colMapper, dialect, tagId) + parentTable := engine.mapType(fieldValue) + for _, col := range parentTable.Columns() { + col.FieldName = fmt.Sprintf("%v.%v", t.Field(i).Name, col.FieldName) + //fmt.Println("---", col.FieldName) + table.AddColumn(col) + } + + continue + } else if fieldValue.Kind() == reflect.Ptr { + f := fieldValue.Type().Elem() + if f.Kind() == reflect.Struct { + fieldValue = fieldValue.Elem() + if !fieldValue.IsValid() || fieldValue.IsNil() { + fieldValue = reflect.New(f).Elem() + } + //fmt.Println("00000", fieldValue) + } + + parentTable := engine.mapType(fieldValue) + for _, col := range parentTable.Columns() { + col.FieldName = fmt.Sprintf("%v.%v", t.Field(i).Name, col.FieldName) + table.AddColumn(col) + } + + continue } - - continue + //TODO: warning } indexNames := make(map[string]int) @@ -599,20 +686,30 @@ func (engine *Engine) mapType(v reflect.Value) *core.Table { continue } col.SQLType = core.SQLType{fs[0], 0, 0} - fs2 := strings.Split(fs[1][0:len(fs[1])-1], ",") - if len(fs2) == 2 { - col.Length, err = strconv.Atoi(fs2[0]) - if err != nil { - engine.LogError(err) + if fs[0] == core.Enum && fs[1][0] == '\'' { //enum + options := strings.Split(fs[1][0:len(fs[1])-1], ",") + col.EnumOptions = make(map[string]int) + for k, v := range options { + v = strings.TrimSpace(v) + v = strings.Trim(v, "'") + col.EnumOptions[v] = k } - col.Length2, err = strconv.Atoi(fs2[1]) - if err != nil { - engine.LogError(err) - } - } else if len(fs2) == 1 { - col.Length, err = strconv.Atoi(fs2[0]) - if err != nil { - engine.LogError(err) + } else { + fs2 := strings.Split(fs[1][0:len(fs[1])-1], ",") + if len(fs2) == 2 { + col.Length, err = strconv.Atoi(fs2[0]) + if err != nil { + engine.LogError(err) + } + col.Length2, err = strconv.Atoi(fs2[1]) + if err != nil { + engine.LogError(err) + } + } else if len(fs2) == 1 { + col.Length, err = strconv.Atoi(fs2[0]) + if err != nil { + engine.LogError(err) + } } } } else { @@ -651,7 +748,17 @@ func (engine *Engine) mapType(v reflect.Value) *core.Table { } } } else { - sqlType := core.Type2SQLType(fieldType) + var sqlType core.SQLType + if fieldValue.CanAddr() { + if _, ok := fieldValue.Addr().Interface().(core.Conversion); ok { + sqlType = core.SQLType{core.Text, 0, 0} + } + } + if _, ok := fieldValue.Interface().(core.Conversion); ok { + sqlType = core.SQLType{core.Text, 0, 0} + } else { + sqlType = core.Type2SQLType(fieldType) + } col = core.NewColumn(engine.ColumnMapper.Obj2Table(t.Field(i).Name), t.Field(i).Name, sqlType, sqlType.DefaultLength, sqlType.DefaultLength2, true) @@ -701,19 +808,25 @@ func (engine *Engine) IsTableEmpty(bean interface{}) (bool, error) { session := engine.NewSession() defer session.Close() rows, err := session.Count(bean) - return rows > 0, err + return rows == 0, err } // If a table is exist func (engine *Engine) IsTableExist(bean interface{}) (bool, error) { v := rValue(bean) - if v.Type().Kind() != reflect.Struct { + var tableName string + if v.Type().Kind() == reflect.String { + tableName = bean.(string) + } else if v.Type().Kind() == reflect.Struct { + table := engine.autoMapType(v) + tableName = table.Name + } else { return false, errors.New("bean should be a struct or struct's point") } - table := engine.autoMapType(v) + session := engine.NewSession() defer session.Close() - has, err := session.isTableExist(table.Name) + has, err := session.isTableExist(tableName) return has, err } @@ -836,7 +949,7 @@ func (engine *Engine) Sync(beans ...interface{}) error { session := engine.NewSession() session.Statement.RefTable = table defer session.Close() - isExist, err := session.isColumnExist(table.Name, col.Name) + isExist, err := session.isColumnExist(table.Name, col) if err != nil { return err } @@ -1120,7 +1233,11 @@ func (engine *Engine) FormatTime(sqlTypeName string, t time.Time) (v interface{} case core.Date: v = engine.TZTime(t).Format("2006-01-02") case core.DateTime, core.TimeStamp: - v = engine.TZTime(t).Format("2006-01-02 15:04:05") + if engine.dialect.DBType() == "ql" { + v = engine.TZTime(t) + } else { + v = engine.TZTime(t).Format("2006-01-02 15:04:05") + } case core.TimeStampz: if engine.dialect.DBType() == core.MSSQL { v = engine.TZTime(t).Format("2006-01-02T15:04:05.9999999Z07:00") diff --git a/examples/sync.go b/examples/sync.go index ad28ad80..d108e455 100644 --- a/examples/sync.go +++ b/examples/sync.go @@ -88,6 +88,17 @@ func main() { _, err = Orm.Insert(user) if err != nil { fmt.Println(err) + return + } + + isexist, err := Orm.IsTableExist("sync_user2") + if err != nil { + fmt.Println(err) + return + } + if !isexist { + fmt.Println("sync_user2 is not exist") + return } } } diff --git a/examples/tables.go b/examples/tables.go new file mode 100644 index 00000000..97c842be --- /dev/null +++ b/examples/tables.go @@ -0,0 +1,34 @@ +package main + +import ( + "fmt" + "os" + + "github.com/go-xorm/xorm" + _ "github.com/mattn/go-sqlite3" +) + +func main() { + if len(os.Args) < 2 { + fmt.Println("need db path") + return + } + + orm, err := xorm.NewEngine("sqlite3", os.Args[1]) + if err != nil { + fmt.Println(err) + return + } + defer orm.Close() + orm.ShowSQL = true + + tables, err := orm.DBMetas() + if err != nil { + fmt.Println(err) + return + } + + for _, table := range tables { + fmt.Println(table.Name) + } +} diff --git a/helpers.go b/helpers.go index fcf3c21e..71b29074 100644 --- a/helpers.go +++ b/helpers.go @@ -3,6 +3,7 @@ package xorm import ( "fmt" "reflect" + "sort" "strconv" "strings" "time" @@ -56,24 +57,20 @@ func structName(v reflect.Type) string { } func sliceEq(left, right []string) bool { - for _, l := range left { - var find bool - for _, r := range right { - if l == r { - find = true - break - } - } - if !find { + if len(left) != len(right) { + return false + } + sort.Sort(sort.StringSlice(left)) + sort.Sort(sort.StringSlice(right)) + for i := 0; i < len(left); i++ { + if left[i] != right[i] { return false } } - return true } func value2Bytes(rawValue *reflect.Value) (data []byte, err error) { - aa := reflect.TypeOf((*rawValue).Interface()) vv := reflect.ValueOf((*rawValue).Interface()) diff --git a/mssql_dialect.go b/mssql_dialect.go index 292bb67c..ebc35164 100644 --- a/mssql_dialect.go +++ b/mssql_dialect.go @@ -106,10 +106,16 @@ func (db *mssql) IndexCheckSql(tableName, idxName string) (string, []interface{} return sql, args } -func (db *mssql) ColumnCheckSql(tableName, colName string) (string, []interface{}) { +/*func (db *mssql) ColumnCheckSql(tableName, colName string) (string, []interface{}) { args := []interface{}{tableName, colName} sql := `SELECT "COLUMN_NAME" FROM "INFORMATION_SCHEMA"."COLUMNS" WHERE "TABLE_NAME" = ? AND "COLUMN_NAME" = ?` return sql, args +}*/ + +func (db *mssql) IsColumnExist(tableName string, col *core.Column) (bool, error) { + query := `SELECT "COLUMN_NAME" FROM "INFORMATION_SCHEMA"."COLUMNS" WHERE "TABLE_NAME" = ? AND "COLUMN_NAME" = ?` + + return db.HasRecords(query, tableName, col.Name) } func (db *mssql) TableCheckSql(tableName string) (string, []interface{}) { @@ -128,6 +134,8 @@ where a.object_id=object_id('` + tableName + `')` if err != nil { return nil, nil, err } + defer rows.Close() + cols := make(map[string]*core.Column) colSeq := make([]string, 0) for rows.Next() { @@ -183,6 +191,7 @@ func (db *mssql) GetTables() ([]*core.Table, error) { if err != nil { return nil, err } + defer rows.Close() tables := make([]*core.Table, 0) for rows.Next() { @@ -211,17 +220,19 @@ INNER JOIN SYS.COLUMNS C ON IXS.OBJECT_ID=C.OBJECT_ID AND IXCS.COLUMN_ID=C.COLUMN_ID WHERE IXS.TYPE_DESC='NONCLUSTERED' and OBJECT_NAME(IXS.OBJECT_ID) =? ` + rows, err := db.DB().Query(s, args...) if err != nil { return nil, err } + defer rows.Close() indexes := make(map[string]*core.Index, 0) for rows.Next() { var indexType int var indexName, colName, isUnique string - err = rows.Scan(&indexName, &colName, &isUnique, nil) + err = rows.Scan(&indexName, &colName, &isUnique) if err != nil { return nil, err } @@ -262,7 +273,7 @@ func (db *mssql) CreateTableSql(table *core.Table, tableName, storeEngine, chars tableName = table.Name } - sql = "IF NOT EXISTS (SELECT [name] FROM sys.tables WHERE [name] = '" + tableName + "' ) CREATE TABLE" + sql = "IF NOT EXISTS (SELECT [name] FROM sys.tables WHERE [name] = '" + tableName + "' ) CREATE TABLE " sql += db.QuoteStr() + tableName + db.QuoteStr() + " (" diff --git a/mysql_dialect.go b/mysql_dialect.go index 71273183..e76830ab 100644 --- a/mysql_dialect.go +++ b/mysql_dialect.go @@ -53,6 +53,17 @@ func (db *mysql) SqlType(c *core.Column) string { case core.TimeStampz: res = core.Char c.Length = 64 + case core.Enum: //mysql enum + res = core.Enum + res += "(" + for v, k := range c.EnumOptions { + if k > 0 { + res += fmt.Sprintf(",'%v'", v) + } else { + res += fmt.Sprintf("'%v'", v) + } + } + res += ")" default: res = t } @@ -98,11 +109,11 @@ func (db *mysql) IndexCheckSql(tableName, idxName string) (string, []interface{} return sql, args } -func (db *mysql) ColumnCheckSql(tableName, colName string) (string, []interface{}) { +/*func (db *mysql) ColumnCheckSql(tableName, colName string) (string, []interface{}) { args := []interface{}{db.DbName, tableName, colName} sql := "SELECT `COLUMN_NAME` FROM `INFORMATION_SCHEMA`.`COLUMNS` WHERE `TABLE_SCHEMA` = ? AND `TABLE_NAME` = ? AND `COLUMN_NAME` = ?" return sql, args -} +}*/ func (db *mysql) TableCheckSql(tableName string) (string, []interface{}) { args := []interface{}{db.DbName, tableName} @@ -140,26 +151,39 @@ func (db *mysql) GetColumns(tableName string) ([]string, map[string]*core.Column if colDefault != nil { col.Default = *colDefault + if col.Default == "" { + col.DefaultIsEmpty = true + } } cts := strings.Split(colType, "(") + colName := cts[0] + colType = strings.ToUpper(colName) var len1, len2 int if len(cts) == 2 { idx := strings.Index(cts[1], ")") - lens := strings.Split(cts[1][0:idx], ",") - len1, err = strconv.Atoi(strings.TrimSpace(lens[0])) - if err != nil { - return nil, nil, err - } - if len(lens) == 2 { - len2, err = strconv.Atoi(lens[1]) + if colType == core.Enum && cts[1][0] == '\'' { //enum + options := strings.Split(cts[1][0:idx], ",") + col.EnumOptions = make(map[string]int) + for k, v := range options { + v = strings.TrimSpace(v) + v = strings.Trim(v, "'") + col.EnumOptions[v] = k + } + } else { + lens := strings.Split(cts[1][0:idx], ",") + len1, err = strconv.Atoi(strings.TrimSpace(lens[0])) if err != nil { return nil, nil, err } + if len(lens) == 2 { + len2, err = strconv.Atoi(lens[1]) + if err != nil { + return nil, nil, err + } + } } } - colName := cts[0] - colType = strings.ToUpper(colName) col.Length = len1 col.Length2 = len2 if _, ok := core.SqlTypes[colType]; ok { @@ -182,6 +206,10 @@ func (db *mysql) GetColumns(tableName string) ([]string, map[string]*core.Column if col.SQLType.IsText() { if col.Default != "" { col.Default = "'" + col.Default + "'" + } else { + if col.DefaultIsEmpty { + col.Default = "''" + } } } cols[col.Name] = col diff --git a/oracle_dialect.go b/oracle_dialect.go index b22db57a..73300c1a 100644 --- a/oracle_dialect.go +++ b/oracle_dialect.go @@ -87,10 +87,26 @@ func (db *oracle) TableCheckSql(tableName string) (string, []interface{}) { return `SELECT table_name FROM user_tables WHERE table_name = ?`, args } -func (db *oracle) ColumnCheckSql(tableName, colName string) (string, []interface{}) { +/*func (db *oracle) ColumnCheckSql(tableName, colName string) (string, []interface{}) { args := []interface{}{strings.ToUpper(tableName), strings.ToUpper(colName)} return "SELECT column_name FROM USER_TAB_COLUMNS WHERE table_name = ?" + " AND column_name = ?", args +}*/ + +func (db *oracle) IsColumnExist(tableName string, col *core.Column) (bool, error) { + args := []interface{}{strings.ToUpper(tableName), strings.ToUpper(col.Name)} + query := "SELECT column_name FROM USER_TAB_COLUMNS WHERE table_name = ?" + + " AND column_name = ?" + rows, err := db.DB().Query(query, args...) + if err != nil { + return false, err + } + defer rows.Close() + + if rows.Next() { + return true, nil + } + return false, nil } func (db *oracle) GetColumns(tableName string) ([]string, map[string]*core.Column, error) { diff --git a/postgres_dialect.go b/postgres_dialect.go index 943039e5..58e8da4f 100644 --- a/postgres_dialect.go +++ b/postgres_dialect.go @@ -102,10 +102,28 @@ func (db *postgres) TableCheckSql(tableName string) (string, []interface{}) { return `SELECT tablename FROM pg_tables WHERE tablename = ?`, args } -func (db *postgres) ColumnCheckSql(tableName, colName string) (string, []interface{}) { +/*func (db *postgres) ColumnCheckSql(tableName, colName string) (string, []interface{}) { args := []interface{}{tableName, colName} return "SELECT column_name FROM INFORMATION_SCHEMA.COLUMNS WHERE table_name = ?" + " AND column_name = ?", args +}*/ + +func (db *postgres) IsColumnExist(tableName string, col *core.Column) (bool, error) { + args := []interface{}{tableName, col.Name} + + //query := "SELECT `COLUMN_NAME` FROM `INFORMATION_SCHEMA`.`COLUMNS` WHERE `TABLE_SCHEMA` = ? AND `TABLE_NAME` = ? AND `COLUMN_NAME` = ?" + query := "SELECT column_name FROM INFORMATION_SCHEMA.COLUMNS WHERE table_name = $1" + + " AND column_name = $2" + rows, err := db.DB().Query(query, args...) + if err != nil { + return false, err + } + defer rows.Close() + + if rows.Next() { + return true, nil + } + return false, nil } func (db *postgres) GetColumns(tableName string) ([]string, map[string]*core.Column, error) { diff --git a/processors.go b/processors.go index 770515e6..03ae8e0f 100644 --- a/processors.go +++ b/processors.go @@ -15,6 +15,10 @@ type BeforeDeleteProcessor interface { BeforeDelete() } +type BeforeSetProcessor interface { + BeforeSet(string, Cell) +} + // !nashtsai! TODO enable BeforeValidateProcessor when xorm start to support validations //// Executed before an object is validated //type BeforeValidateProcessor interface { diff --git a/session.go b/session.go index feeda785..68cd8d22 100644 --- a/session.go +++ b/session.go @@ -227,7 +227,7 @@ func (session *Session) StoreEngine(storeEngine string) *Session { return session } -// Method StoreEngine is only avialble charset dialect currently +// Method Charset is only avialble mysql dialect currently func (session *Session) Charset(charset string) *Session { session.Statement.Charset = charset return session @@ -718,7 +718,7 @@ func (session *Session) cacheFind(t reflect.Type, sqlStr string, rowsSlicePtr in if err != nil { return err } - // 查询数目太大，采用缓存将不是一个很好的方式。 + // 查询数目太大，采用缓存将不是一个很好的方式〠if len(resultsSlice) > 500 { session.Engine.LogDebug("[xorm:cacheFind] ids length %v > 500, no cache", len(resultsSlice)) return ErrCacheFailed @@ -883,7 +883,6 @@ func (session *Session) Rows(bean interface{}) (*Rows, error) { // are conditions. beans could be []Struct, []*Struct, map[int64]Struct // map[int64]*Struct func (session *Session) Iterate(bean interface{}, fun IterFunc) error { - rows, err := session.Rows(bean) if err != nil { return err @@ -982,24 +981,6 @@ func (session *Session) Get(bean interface{}) (bool, error) { } else { return false, nil } - - // resultsSlice, err := session.query(sqlStr, args...) - // if err != nil { - // return false, err - // } - // if len(resultsSlice) < 1 { - // return false, nil - // } - - // err = session.scanMapIntoStruct(bean, resultsSlice[0]) - // if err != nil { - // return true, err - // } - // if len(resultsSlice) == 1 { - // return true, nil - // } else { - // return true, errors.New("More than one record") - // } } // Count counts the records. bean's non-empty fields @@ -1253,7 +1234,7 @@ func (session *Session) Ping() error { return session.Db.Ping() } -func (session *Session) isColumnExist(tableName, colName string) (bool, error) { +func (session *Session) isColumnExist(tableName string, col *core.Column) (bool, error) { err := session.newDb() if err != nil { return false, err @@ -1262,9 +1243,10 @@ func (session *Session) isColumnExist(tableName, colName string) (bool, error) { if session.IsAutoClose { defer session.Close() } - sqlStr, args := session.Engine.dialect.ColumnCheckSql(tableName, colName) - results, err := session.query(sqlStr, args...) - return len(results) > 0, err + return session.Engine.dialect.IsColumnExist(tableName, col) + //sqlStr, args := session.Engine.dialect.ColumnCheckSql(tableName, colName) + //results, err := session.query(sqlStr, args...) + //return len(results) > 0, err } func (session *Session) isTableExist(tableName string) (bool, error) { @@ -1441,6 +1423,8 @@ func (session *Session) getField(dataStruct *reflect.Value, key string, table *c return fieldValue } +type Cell *interface{} + func (session *Session) row2Bean(rows *core.Rows, fields []string, fieldsCount int, bean interface{}) error { dataStruct := rValue(bean) if dataStruct.Kind() != reflect.Struct { @@ -1449,18 +1433,24 @@ func (session *Session) row2Bean(rows *core.Rows, fields []string, fieldsCount i table := session.Engine.autoMapType(dataStruct) - scanResultContainers := make([]interface{}, len(fields)) + scanResults := make([]interface{}, len(fields)) for i := 0; i < len(fields); i++ { - var scanResultContainer interface{} - scanResultContainers[i] = &scanResultContainer + var cell interface{} + scanResults[i] = &cell } - if err := rows.Scan(scanResultContainers...); err != nil { + if err := rows.Scan(scanResults...); err != nil { return err } + if b, hasBeforeSet := bean.(BeforeSetProcessor); hasBeforeSet { + for ii, key := range fields { + b.BeforeSet(key, Cell(scanResults[ii].(*interface{}))) + } + } + for ii, key := range fields { if fieldValue := session.getField(&dataStruct, key, table); fieldValue != nil { - rawValue := reflect.Indirect(reflect.ValueOf(scanResultContainers[ii])) + rawValue := reflect.Indirect(reflect.ValueOf(scanResults[ii])) //if row is null then ignore if rawValue.Interface() == nil { @@ -1468,9 +1458,23 @@ func (session *Session) row2Bean(rows *core.Rows, fields []string, fieldsCount i continue } - if structConvert, ok := fieldValue.Addr().Interface().(core.Conversion); ok { + if fieldValue.CanAddr() { + if structConvert, ok := fieldValue.Addr().Interface().(core.Conversion); ok { + if data, err := value2Bytes(&rawValue); err == nil { + structConvert.FromDB(data) + } else { + session.Engine.LogError(err) + } + continue + } + } + + if _, ok := fieldValue.Interface().(core.Conversion); ok { if data, err := value2Bytes(&rawValue); err == nil { - structConvert.FromDB(data) + if fieldValue.Kind() == reflect.Ptr && fieldValue.IsNil() { + fieldValue.Set(reflect.New(fieldValue.Type().Elem())) + } + fieldValue.Interface().(core.Conversion).FromDB(data) } else { session.Engine.LogError(err) } @@ -2057,6 +2061,10 @@ func (session *Session) bytes2Value(col *core.Column, fieldValue *reflect.Value, return structConvert.FromDB(data) } + if structConvert, ok := fieldValue.Interface().(core.Conversion); ok { + return structConvert.FromDB(data) + } + var v interface{} key := col.Name fieldType := fieldValue.Type() @@ -2451,6 +2459,16 @@ func (session *Session) value2Interface(col *core.Column, fieldValue reflect.Val } } } + + if fieldConvert, ok := fieldValue.Interface().(core.Conversion); ok { + data, err := fieldConvert.ToDB() + if err != nil { + return 0, err + } else { + return string(data), nil + } + } + fieldType := fieldValue.Type() k := fieldType.Kind() if k == reflect.Ptr { @@ -2470,24 +2488,19 @@ func (session *Session) value2Interface(col *core.Column, fieldValue reflect.Val switch k { case reflect.Bool: return fieldValue.Bool(), nil - /*if fieldValue.Bool() { - return 1, nil - } else { - return 0, nil - }*/ case reflect.String: return fieldValue.String(), nil case reflect.Struct: if fieldType == core.TimeType { - t := fieldValue.Interface().(time.Time) - if session.Engine.dialect.DBType() == core.MSSQL { - if t.IsZero() { - return nil, nil - } - } switch fieldValue.Interface().(type) { case time.Time: - tf := session.Engine.FormatTime(col.SQLType.Name, fieldValue.Interface().(time.Time)) + t := fieldValue.Interface().(time.Time) + if session.Engine.dialect.DBType() == core.MSSQL { + if t.IsZero() { + return nil, nil + } + } + tf := session.Engine.FormatTime(col.SQLType.Name, t) return tf, nil default: return fieldValue.Interface(), nil @@ -2947,9 +2960,9 @@ func (session *Session) Update(bean interface{}, condiBean ...interface{}) (int6 session.Statement.RefTable = table if session.Statement.ColumnStr == "" { - colNames, args = buildConditions(session.Engine, table, bean, false, false, + colNames, args = buildUpdates(session.Engine, table, bean, false, false, false, false, session.Statement.allUseBool, session.Statement.useAllCols, - session.Statement.mustColumnMap) + session.Statement.mustColumnMap, true) } else { colNames, args, err = genCols(table, session, bean, true, true) if err != nil { @@ -3003,11 +3016,12 @@ func (session *Session) Update(bean interface{}, condiBean ...interface{}) (int6 if condition == "" { if len(condiColNames) > 0 { - condition = fmt.Sprintf("%v", strings.Join(condiColNames, " AND ")) + condition = fmt.Sprintf("%v", strings.Join(condiColNames, " "+session.Engine.Dialect().AndStr()+" ")) } } else { if len(condiColNames) > 0 { - condition = fmt.Sprintf("(%v) AND (%v)", condition, strings.Join(condiColNames, " AND ")) + condition = fmt.Sprintf("(%v) %v (%v)", condition, + session.Engine.Dialect().AndStr(), strings.Join(condiColNames, " "+session.Engine.Dialect().AndStr()+" ")) } } @@ -3017,7 +3031,7 @@ func (session *Session) Update(bean interface{}, condiBean ...interface{}) (int6 var verValue *reflect.Value if table.Version != "" && session.Statement.checkVersion { if condition != "" { - condition = fmt.Sprintf("WHERE (%v) AND %v = ?", condition, + condition = fmt.Sprintf("WHERE (%v) %v %v = ?", condition, session.Engine.Dialect().AndStr(), session.Engine.Quote(table.Version)) } else { condition = fmt.Sprintf("WHERE %v = ?", session.Engine.Quote(table.Version)) @@ -3025,7 +3039,7 @@ func (session *Session) Update(bean interface{}, condiBean ...interface{}) (int6 inSql, inArgs = session.Statement.genInSql() if len(inSql) > 0 { if condition != "" { - condition += " AND " + inSql + condition += " " + session.Engine.Dialect().AndStr() + " " + inSql } else { condition = "WHERE " + inSql } @@ -3051,7 +3065,7 @@ func (session *Session) Update(bean interface{}, condiBean ...interface{}) (int6 inSql, inArgs = session.Statement.genInSql() if len(inSql) > 0 { if condition != "" { - condition += " AND " + inSql + condition += " " + session.Engine.Dialect().AndStr() + " " + inSql } else { condition = "WHERE " + inSql } @@ -3196,20 +3210,21 @@ func (session *Session) Delete(bean interface{}) (int64, error) { session.Statement.mustColumnMap) var condition = "" + var andStr = session.Engine.dialect.AndStr() session.Statement.processIdParam() if session.Statement.WhereStr != "" { condition = session.Statement.WhereStr if len(colNames) > 0 { - condition += " AND " + strings.Join(colNames, " AND ") + condition += " " + andStr + " " + strings.Join(colNames, " "+andStr+" ") } } else { - condition = strings.Join(colNames, " AND ") + condition = strings.Join(colNames, " "+andStr+" ") } inSql, inArgs := session.Statement.genInSql() if len(inSql) > 0 { if len(condition) > 0 { - condition += " AND " + condition += " " + andStr + " " } condition += inSql args = append(args, inArgs...) @@ -3314,7 +3329,7 @@ func genCols(table *core.Table, session *Session, bean interface{}, useCol bool, } if (col.IsCreated || col.IsUpdated) && session.Statement.UseAutoTime { - args = append(args, time.Now()) + args = append(args, session.Engine.NowTime(col.SQLType.Name)) } else if col.IsVersion && session.Statement.checkVersion { args = append(args, 1) } else { diff --git a/sqlite3_dialect.go b/sqlite3_dialect.go index 0e19f96c..c6123dcb 100644 --- a/sqlite3_dialect.go +++ b/sqlite3_dialect.go @@ -1,6 +1,7 @@ package xorm import ( + "fmt" "strings" "github.com/go-xorm/core" @@ -44,6 +45,10 @@ func (db *sqlite3) SqlType(c *core.Column) string { } } +func (db *sqlite3) FormatBytes(bs []byte) string { + return fmt.Sprintf("X'%x'", bs) +} + func (db *sqlite3) SupportInsertMany() bool { return true } @@ -78,10 +83,25 @@ func (db *sqlite3) TableCheckSql(tableName string) (string, []interface{}) { return "SELECT name FROM sqlite_master WHERE type='table' and name = ?", args } -func (db *sqlite3) ColumnCheckSql(tableName, colName string) (string, []interface{}) { +/*func (db *sqlite3) ColumnCheckSql(tableName, colName string) (string, []interface{}) { args := []interface{}{tableName} sql := "SELECT name FROM sqlite_master WHERE type='table' and name = ? and ((sql like '%`" + colName + "`%') or (sql like '%[" + colName + "]%'))" return sql, args +}*/ + +func (db *sqlite3) IsColumnExist(tableName string, col *core.Column) (bool, error) { + args := []interface{}{tableName} + query := "SELECT name FROM sqlite_master WHERE type='table' and name = ? and ((sql like '%`" + col.Name + "`%') or (sql like '%[" + col.Name + "]%'))" + rows, err := db.DB().Query(query, args...) + if err != nil { + return false, err + } + defer rows.Close() + + if rows.Next() { + return true, nil + } + return false, nil } func (db *sqlite3) GetColumns(tableName string) ([]string, map[string]*core.Column, error) { diff --git a/statement.go b/statement.go index 2de1f84a..707f9c1d 100644 --- a/statement.go +++ b/statement.go @@ -96,6 +96,9 @@ func (statement *Statement) Sql(querystring string, args ...interface{}) *Statem // add Where statment func (statement *Statement) Where(querystring string, args ...interface{}) *Statement { + if !strings.Contains(querystring, statement.Engine.dialect.EqStr()) { + querystring = strings.Replace(querystring, "=", statement.Engine.dialect.EqStr(), -1) + } statement.WhereStr = querystring statement.Params = args return statement @@ -254,10 +257,10 @@ func (statement *Statement) Table(tableNameOrBean interface{}) *Statement { }*/ // Auto generating conditions according a struct -func buildConditions(engine *Engine, table *core.Table, bean interface{}, +func buildUpdates(engine *Engine, table *core.Table, bean interface{}, includeVersion bool, includeUpdated bool, includeNil bool, includeAutoIncr bool, allUseBool bool, useAllCols bool, - mustColumnMap map[string]bool) ([]string, []interface{}) { + mustColumnMap map[string]bool, update bool) ([]string, []interface{}) { colNames := make([]string, 0) var args = make([]interface{}, 0) @@ -265,6 +268,9 @@ func buildConditions(engine *Engine, table *core.Table, bean interface{}, if !includeVersion && col.IsVersion { continue } + if col.IsCreated { + continue + } if !includeUpdated && col.IsUpdated { continue } @@ -294,6 +300,30 @@ func buildConditions(engine *Engine, table *core.Table, bean interface{}, } } + var val interface{} + + if fieldValue.CanAddr() { + if structConvert, ok := fieldValue.Addr().Interface().(core.Conversion); ok { + data, err := structConvert.ToDB() + if err != nil { + engine.LogError(err) + } else { + val = data + } + goto APPEND + } + } + + if structConvert, ok := fieldValue.Interface().(core.Conversion); ok { + data, err := structConvert.ToDB() + if err != nil { + engine.LogError(err) + } else { + val = data + } + goto APPEND + } + if fieldType.Kind() == reflect.Ptr { if fieldValue.IsNil() { if includeNil { @@ -311,6 +341,178 @@ func buildConditions(engine *Engine, table *core.Table, bean interface{}, } } + switch fieldType.Kind() { + case reflect.Bool: + if allUseBool || requiredField { + val = fieldValue.Interface() + } else { + // if a bool in a struct, it will not be as a condition because it default is false, + // please use Where() instead + continue + } + case reflect.String: + if !requiredField && fieldValue.String() == "" { + continue + } + // for MyString, should convert to string or panic + if fieldType.String() != reflect.String.String() { + val = fieldValue.String() + } else { + val = fieldValue.Interface() + } + case reflect.Int8, reflect.Int16, reflect.Int, reflect.Int32, reflect.Int64: + if !requiredField && fieldValue.Int() == 0 { + continue + } + val = fieldValue.Interface() + case reflect.Float32, reflect.Float64: + if !requiredField && fieldValue.Float() == 0.0 { + continue + } + val = fieldValue.Interface() + case reflect.Uint8, reflect.Uint16, reflect.Uint, reflect.Uint32, reflect.Uint64: + if !requiredField && fieldValue.Uint() == 0 { + continue + } + val = fieldValue.Interface() + case reflect.Struct: + if fieldType == reflect.TypeOf(time.Now()) { + t := fieldValue.Interface().(time.Time) + if !requiredField && (t.IsZero() || !fieldValue.IsValid()) { + continue + } + val = engine.FormatTime(col.SQLType.Name, t) + //fmt.Println("-------", t, val, col.Name) + } else { + engine.autoMapType(fieldValue) + if table, ok := engine.Tables[fieldValue.Type()]; ok { + if len(table.PrimaryKeys) == 1 { + pkField := reflect.Indirect(fieldValue).FieldByName(table.PKColumns()[0].FieldName) + if pkField.Int() != 0 { + val = pkField.Interface() + } else { + continue + } + } else { + //TODO: how to handler? + } + } else { + val = fieldValue.Interface() + } + } + case reflect.Array, reflect.Slice, reflect.Map: + if fieldValue == reflect.Zero(fieldType) { + continue + } + if fieldValue.IsNil() || !fieldValue.IsValid() || fieldValue.Len() == 0 { + continue + } + + if col.SQLType.IsText() { + bytes, err := json.Marshal(fieldValue.Interface()) + if err != nil { + engine.LogError(err) + continue + } + val = string(bytes) + } else if col.SQLType.IsBlob() { + var bytes []byte + var err error + if (fieldType.Kind() == reflect.Array || fieldType.Kind() == reflect.Slice) && + fieldType.Elem().Kind() == reflect.Uint8 { + if fieldValue.Len() > 0 { + val = fieldValue.Bytes() + } else { + continue + } + } else { + bytes, err = json.Marshal(fieldValue.Interface()) + if err != nil { + engine.LogError(err) + continue + } + val = bytes + } + } else { + continue + } + default: + val = fieldValue.Interface() + } + + APPEND: + //fmt.Println("==", col.Name, "==", fmt.Sprintf("%v", val)) + args = append(args, val) + if col.IsPrimaryKey && engine.dialect.DBType() == "ql" { + continue + } + colNames = append(colNames, fmt.Sprintf("%v = ?", engine.Quote(col.Name))) + } + + return colNames, args +} + +// Auto generating conditions according a struct +func buildConditions(engine *Engine, table *core.Table, bean interface{}, + includeVersion bool, includeUpdated bool, includeNil bool, + includeAutoIncr bool, allUseBool bool, useAllCols bool, + mustColumnMap map[string]bool) ([]string, []interface{}) { + + colNames := make([]string, 0) + var args = make([]interface{}, 0) + for _, col := range table.Columns() { + if !includeVersion && col.IsVersion { + continue + } + if !includeUpdated && col.IsUpdated { + continue + } + if !includeAutoIncr && col.IsAutoIncrement { + continue + } + // + //fmt.Println(engine.dialect.DBType(), Text) + if engine.dialect.DBType() == core.MSSQL && col.SQLType.Name == core.Text { + continue + } + fieldValuePtr, err := col.ValueOf(bean) + if err != nil { + engine.LogError(err) + continue + } + + fieldValue := *fieldValuePtr + if fieldValue.Interface() == nil { + continue + } + + fieldType := reflect.TypeOf(fieldValue.Interface()) + requiredField := useAllCols + if b, ok := mustColumnMap[strings.ToLower(col.Name)]; ok { + if b { + requiredField = true + } else { + continue + } + } + + if fieldType.Kind() == reflect.Ptr { + if fieldValue.IsNil() { + if includeNil { + args = append(args, nil) + colNames = append(colNames, fmt.Sprintf("%v %s ?", engine.Quote(col.Name), engine.dialect.EqStr())) + } + continue + } else if !fieldValue.IsValid() { + continue + } else { + // dereference ptr type to instance type + fieldValue = fieldValue.Elem() + fieldType = reflect.TypeOf(fieldValue.Interface()) + requiredField = true + } + } + var val interface{} switch fieldType.Kind() { case reflect.Bool: @@ -353,7 +555,7 @@ func buildConditions(engine *Engine, table *core.Table, bean interface{}, continue } val = engine.FormatTime(col.SQLType.Name, t) - fmt.Println("-------", t, val, col.Name) + //fmt.Println("-------", t, val, col.Name) } else { engine.autoMapType(fieldValue) if table, ok := engine.Tables[fieldValue.Type()]; ok { @@ -412,7 +614,13 @@ func buildConditions(engine *Engine, table *core.Table, bean interface{}, } args = append(args, val) - colNames = append(colNames, fmt.Sprintf("%v=?", engine.Quote(col.Name))) + var condi string + if col.IsPrimaryKey && engine.dialect.DBType() == "ql" { + condi = "id() == ?" + } else { + condi = fmt.Sprintf("%v %s ?", engine.Quote(col.Name), engine.dialect.EqStr()) + } + colNames = append(colNames, condi) } return colNames, args @@ -476,10 +684,22 @@ func (statement *Statement) getInc() map[string]incrParam { // Generate "Where column IN (?) " statment func (statement *Statement) In(column string, args ...interface{}) *Statement { k := strings.ToLower(column) - if _, ok := statement.inColumns[k]; ok { - statement.inColumns[k].args = append(statement.inColumns[k].args, args...) + var newargs []interface{} + if len(args) == 1 && + reflect.TypeOf(args[0]).Kind() == reflect.Slice { + newargs = make([]interface{}, 0) + v := reflect.ValueOf(args[0]) + for i := 0; i < v.Len(); i++ { + newargs = append(newargs, v.Index(i).Interface()) + } } else { - statement.inColumns[k] = &inParam{column, args} + newargs = args + } + + if _, ok := statement.inColumns[k]; ok { + statement.inColumns[k].args = append(statement.inColumns[k].args, newargs...) + } else { + statement.inColumns[k] = &inParam{column, newargs} } return statement } @@ -646,7 +866,22 @@ func (statement *Statement) genColumnStr() string { if col.MapType == core.ONLYTODB { continue } - colNames = append(colNames, statement.Engine.Quote(statement.TableName())+"."+statement.Engine.Quote(col.Name)) + + if statement.JoinStr != "" { + name := statement.Engine.Quote(statement.TableName()) + "." + statement.Engine.Quote(col.Name) + if col.IsPrimaryKey && statement.Engine.Dialect().DBType() == "ql" { + colNames = append(colNames, "id() as "+name) + } else { + colNames = append(colNames, name) + } + } else { + name := statement.Engine.Quote(col.Name) + if col.IsPrimaryKey && statement.Engine.Dialect().DBType() == "ql" { + colNames = append(colNames, "id() as "+name) + } else { + colNames = append(colNames, name) + } + } } return strings.Join(colNames, ", ") } @@ -728,12 +963,13 @@ func (statement *Statement) genGetSql(bean interface{}) (string, []interface{}) columnStr = statement.genColumnStr() } + statement.attachInSql() // !admpub! fix bug:Iterate func missing "... IN (...)" return statement.genSelectSql(columnStr), append(statement.Params, statement.BeanArgs...) } func (s *Statement) genAddColumnStr(col *core.Column) (string, []interface{}) { quote := s.Engine.Quote - sql := fmt.Sprintf("ALTER TABLE %v ADD COLUMN %v;", quote(s.TableName()), + sql := fmt.Sprintf("ALTER TABLE %v ADD %v;", quote(s.TableName()), col.String(s.Engine.dialect)) return sql, []interface{}{} } @@ -759,14 +995,15 @@ func (statement *Statement) genCountSql(bean interface{}) (string, []interface{} colNames, args := buildConditions(statement.Engine, table, bean, true, true, false, true, statement.allUseBool, statement.useAllCols, statement.mustColumnMap) - statement.ConditionStr = strings.Join(colNames, " AND ") + statement.ConditionStr = strings.Join(colNames, " "+statement.Engine.Dialect().AndStr()+" ") statement.BeanArgs = args + // count(index fieldname) > count(0) > count(*) - var id string = "0" - if len(table.PrimaryKeys) == 1 { - id = statement.Engine.Quote(table.PrimaryKeys[0]) + var id string = "*" + if statement.Engine.Dialect().DBType() == "ql" { + id = "" } - return statement.genSelectSql(fmt.Sprintf("COUNT(%v) AS %v", id, statement.Engine.Quote("total"))), append(statement.Params, statement.BeanArgs...) + return statement.genSelectSql(fmt.Sprintf("count(%v) AS %v", id, statement.Engine.Quote("total"))), append(statement.Params, statement.BeanArgs...) } func (statement *Statement) genSelectSql(columnStr string) (a string) { @@ -779,20 +1016,59 @@ func (statement *Statement) genSelectSql(columnStr string) (a string) { distinct = "DISTINCT " } - // !nashtsai! REVIEW Sprintf is considered slowest mean of string concatnation, better to work with builder pattern - a = fmt.Sprintf("SELECT %v%v FROM %v", distinct, columnStr, - statement.Engine.Quote(statement.TableName())) - if statement.JoinStr != "" { - a = fmt.Sprintf("%v %v", a, statement.JoinStr) + var top string + var mssqlCondi string + var orderBy string + if statement.OrderStr != "" { + orderBy = fmt.Sprintf(" ORDER BY %v", statement.OrderStr) } statement.processIdParam() + var whereStr string if statement.WhereStr != "" { - a = fmt.Sprintf("%v WHERE %v", a, statement.WhereStr) + whereStr = fmt.Sprintf(" WHERE %v", statement.WhereStr) if statement.ConditionStr != "" { - a = fmt.Sprintf("%v AND %v", a, statement.ConditionStr) + whereStr = fmt.Sprintf("%v %s %v", whereStr, statement.Engine.Dialect().AndStr(), + statement.ConditionStr) } } else if statement.ConditionStr != "" { - a = fmt.Sprintf("%v WHERE %v", a, statement.ConditionStr) + whereStr = fmt.Sprintf(" WHERE %v", statement.ConditionStr) + } + var fromStr string = " FROM " + statement.Engine.Quote(statement.TableName()) + if statement.JoinStr != "" { + fromStr = fmt.Sprintf("%v %v", fromStr, statement.JoinStr) + } + + if statement.Engine.dialect.DBType() == core.MSSQL { + if statement.LimitN > 0 { + top = fmt.Sprintf(" TOP %d ", statement.LimitN) + } + if statement.Start > 0 { + var column string = "(id)" + if len(statement.RefTable.PKColumns()) == 0 { + for _, index := range statement.RefTable.Indexes { + if len(index.Cols) == 1 { + column = index.Cols[0] + break + } + } + if len(column) == 0 { + column = statement.RefTable.ColumnsSeq()[0] + } + } + mssqlCondi = fmt.Sprintf("(%s NOT IN (SELECT TOP %d %s%s%s%s))", + column, statement.Start, column, fromStr, whereStr, orderBy) + } + } + + // !nashtsai! REVIEW Sprintf is considered slowest mean of string concatnation, better to work with builder pattern + a = fmt.Sprintf("SELECT %v%v%v%v%v", top, distinct, columnStr, + fromStr, whereStr) + if mssqlCondi != "" { + if whereStr != "" { + a += " AND " + mssqlCondi + } else { + a += " WHERE " + mssqlCondi + } } if statement.GroupByStr != "" { @@ -810,11 +1086,6 @@ func (statement *Statement) genSelectSql(columnStr string) (a string) { } else if statement.LimitN > 0 { a = fmt.Sprintf("%v LIMIT %v", a, statement.LimitN) } - } else { - //TODO: for mssql, should handler limit. - /*SELECT * FROM ( - SELECT *, ROW_NUMBER() OVER (ORDER BY id desc) as row FROM "userinfo" - ) a WHERE row > [start] and row <= [start+limit] order by id desc*/ } return @@ -822,11 +1093,19 @@ func (statement *Statement) genSelectSql(columnStr string) (a string) { func (statement *Statement) processIdParam() { if statement.IdParam != nil { - for i, col := range statement.RefTable.PKColumns() { - if i < len(*(statement.IdParam)) { - statement.And(fmt.Sprintf("%v=?", statement.Engine.Quote(col.Name)), (*(statement.IdParam))[i]) - } else { - statement.And(fmt.Sprintf("%v=?", statement.Engine.Quote(col.Name)), "") + if statement.Engine.dialect.DBType() != "ql" { + for i, col := range statement.RefTable.PKColumns() { + if i < len(*(statement.IdParam)) { + statement.And(fmt.Sprintf("%v %s ?", statement.Engine.Quote(col.Name), + statement.Engine.dialect.EqStr()), (*(statement.IdParam))[i]) + } else { + statement.And(fmt.Sprintf("%v %s ?", statement.Engine.Quote(col.Name), + statement.Engine.dialect.EqStr()), "") + } + } + } else { + if len(*(statement.IdParam)) <= 1 { + statement.And("id() == ?", (*(statement.IdParam))[0]) } } }