Discussion:
NULL followup,,,
(too old to reply)
Glenn Johansen
2008-10-22 18:13:08 UTC
Permalink
Seems that with a blank date ( not null ) dtoc() and dtos() on the blank
date return null.????
bugs???

- glenn
Greg Hill
2008-10-22 19:21:42 UTC
Permalink
Post by Glenn Johansen
Seems that with a blank date ( not null ) dtoc() and dtos() on the blank
date return null.????
bugs???
Glenn,
I don't know if I can answer all your question regarding these nulls but one
way to think of them is that they are there own type. They don't evaluate
to true or false in boolean fields.
For charactor fields the empty() function will work in either case of the
field being null or blank.

I always set the rowset property to allowNullFields = false

Greg Hill
Robert Bravery
2008-10-22 19:52:26 UTC
Permalink
Hi,

Seee my other answer. I don;t believe you can store a blank date, dates
evaluate to an interger. There is no equivalant int value for a blank date.
Therefore I do not believe it is a bug.
Do this simple test in excel. Try and put some arbetory int values in some
cells. Change the cell format to date. You will end up with some strange
date. use a value of 0, and you get a date of 1900/01/00, Formating a blank
cell gives no date, this is what you would ecpect. Trying to format a
negative value into date format gives a error message.
Conclusion, you cannot have a blank date, a date has to have a value, or
null.

Robert
Post by Glenn Johansen
Seems that with a blank date ( not null ) dtoc() and dtos() on the blank
date return null.????
bugs???
- glenn
---
avast! Antivirus: Inbound message clean.
Virus Database (VPS): 081022-1, 22/10/2008
Tested on: 22/10/2008 09:46:08 PM
avast! - copyright (c) 1988-2008 ALWIL Software.
http://www.avast.com
--
Web Development, Hosting, Design and Content Management Systems
http://www.integralwebsolutions.co.za



---
avast! Antivirus: Outbound message clean.
Virus Database (VPS): 081022-1, 22/10/2008
Tested on: 22/10/2008 09:52:27 PM
avast! - copyright (c) 1988-2008 ALWIL Software.
http://www.avast.com
Mervyn Bick
2008-10-22 21:06:06 UTC
Permalink
Post by Robert Bravery
Hi,
Seee my other answer. I don;t believe you can store a blank date, dates
evaluate to an interger. There is no equivalant int value for a blank
date. Therefore I do not believe it is a bug.
Do this simple test in excel. Try and put some arbetory int values in
some cells. Change the cell format to date. You will end up with some
strange date. use a value of 0, and you get a date of 1900/01/00,
Formating a blank cell gives no date, this is what you would ecpect.
Trying to format a negative value into date format gives a error message.
Conclusion, you cannot have a blank date, a date has to have a value, or
null.
A date is stored on disk not as a number but as 8 characters for yyyymmdd
irrespective of what the date format is set to for display purposes.
Adding a blank record places 8 spaces (hex 20) in the date position.
Replacing the date with ctod(" / / ") leaves 8 spaces on the disk.
Whether they are the same 8 spaces as before or whether they are new
spaces doesn't really matter, does it?

When dBase reads a date from disk it converts the 8 characters into a
number internally. If those characters are spaces dBase has no way of
telling whether they represent a blank or a null value so, for date fields
anyway, they are the same.

As an aside, numbers are also stored as characters, 1 character per digit
and . (or perhaps , depending on regional settings) for the decimal point


Mervyn.
Robert Bravery
2008-10-22 23:29:18 UTC
Permalink
Mervyn,
I believe youre wrong here. If youre talking about the date data type. But
if your talking about a nornaml char data type. well then thats different.
But I believe the thread is about date data type.

If you want to be specific, data as a data type is NOT stored as character.
But as bytes. Infact all data is stored as bytes. As far as I know you
cannot store a blank byte. It has to represent a byte. My answer was
simplistic, but enough to answer the question.
Most places that I read, all say the same, and that is that dates are stored
as byte intergers, and not characters. When last did you store a Blank
Interger. It is either null or a int value.
From Microsoft:
"Date variables are stored as IEEE 64-bit (8-byte) integers that represent
dates ranging from January 1 of the year 1 through December 31 of the year
9999, and times from 0:00:00 (midnight) through 11:59:59 PM"
It seems that you CANNOT store a blank interger, by nature of the datatype.
By nature there is always something stored, either a null value, which is in
question here, or an interger value. For Date data type this is a value of
0-2958465. Which gives you a date value of between 1900/01/00 and
9999/12/31. But strictly speaking 1900/01/00 is not a valid date. So the
actual valid interger value is 1-2958465. -1 gives an error, as does
2958466. It is not spaces or blanks. It is either a Null or an interger
value, infact it is an 8 byte value.

Robert
Post by Mervyn Bick
Post by Robert Bravery
Hi,
Seee my other answer. I don;t believe you can store a blank date, dates
evaluate to an interger. There is no equivalant int value for a blank
date. Therefore I do not believe it is a bug.
Do this simple test in excel. Try and put some arbetory int values in
some cells. Change the cell format to date. You will end up with some
strange date. use a value of 0, and you get a date of 1900/01/00,
Formating a blank cell gives no date, this is what you would ecpect.
Trying to format a negative value into date format gives a error message.
Conclusion, you cannot have a blank date, a date has to have a value, or
null.
A date is stored on disk not as a number but as 8 characters for yyyymmdd
irrespective of what the date format is set to for display purposes.
Adding a blank record places 8 spaces (hex 20) in the date position.
Replacing the date with ctod(" / / ") leaves 8 spaces on the disk.
Whether they are the same 8 spaces as before or whether they are new
spaces doesn't really matter, does it?
When dBase reads a date from disk it converts the 8 characters into a
number internally. If those characters are spaces dBase has no way of
telling whether they represent a blank or a null value so, for date fields
anyway, they are the same.
As an aside, numbers are also stored as characters, 1 character per digit
and . (or perhaps , depending on regional settings) for the decimal point
Mervyn.
---
avast! Antivirus: Inbound message clean.
Virus Database (VPS): 081022-1, 22/10/2008
Tested on: 23/10/2008 01:06:44 AM
avast! - copyright (c) 1988-2008 ALWIL Software.
http://www.avast.com
--
Web Development, Hosting, Design and Content Management Systems
http://www.integralwebsolutions.co.za



---
avast! Antivirus: Outbound message clean.
Virus Database (VPS): 081022-1, 22/10/2008
Tested on: 23/10/2008 01:29:20 AM
avast! - copyright (c) 1988-2008 ALWIL Software.
http://www.avast.com
Mervyn Bick
2008-10-23 06:35:27 UTC
Permalink
Post by Robert Bravery
Mervyn,
I believe youre wrong here. If youre talking about the date data type.
But if your talking about a nornaml char data type. well then thats
different. But I believe the thread is about date data type.
If you want to be specific, data as a data type is NOT stored as
character. But as bytes. Infact all data is stored as bytes. As far as I
know you cannot store a blank byte. It has to represent a byte. My
answer was simplistic, but enough to answer the question.
Most places that I read, all say the same, and that is that dates are
stored as byte intergers, and not characters. When last did you store a
Blank Interger. It is either null or a int value.
"Date variables are stored as IEEE 64-bit (8-byte) integers that
represent dates ranging from January 1 of the year 1 through December 31
of the year 9999, and times from 0:00:00 (midnight) through 11:59:59 PM"
It seems that you CANNOT store a blank interger, by nature of the
datatype. By nature there is always something stored, either a null
value, which is in question here, or an interger value. For Date data
type this is a value of 0-2958465. Which gives you a date value of
between 1900/01/00 and 9999/12/31. But strictly speaking 1900/01/00 is
not a valid date. So the actual valid interger value is 1-2958465. -1
gives an error, as does 2958466. It is not spaces or blanks. It is
either a Null or an interger value, infact it is an 8 byte value.
Firstly, the first 6 words of my previous post were "A date is stored on
disk" and the second paragraph started with "When dBase reads a date from
disk it converts the 8 characters into a number internally." That
internal number could well be your IEEE 64-bit number. It was not, and
still isn't, important to the point I was making.

I was technically incorrect when I said data was stored as characters. As
you have pointed out that should have been bytes. Fair enough but for
today's date those 8 bytes would have the hex values of 32 30 30 38 31 30
32 33 which represent the ANSI values for 2 0 0 8 1 0 2 3 and this is true
whether dBase and/or regional settings dictate that the date be displayed
as 10/23/2008 or 23/10/2008 or 23-10-2008 or whatever.

You don't believe me? Fine but go and conduct your own little experiment
and then we can talk again. Create a table with a date field and, for
good measure, a numeric field. Append a blank. Replace the date with
date() and the the numeric field with a value such as 12345.67. Append a
blank Append a blank and replace the date and numeric fields as above.
(This is to establish a pattern so that you can easily see where the
records are and what is in them without having to delve too deeply into
the structure of a .dbf table or having to bother with counting bytes.)
Close the table and open it in a hex editor.

The unused date field will have 8 bytes of hex 20 which is the ANSI code
for "space". The unused numeric field will be filled with hex 00 which is
the ANSI code for null. If a numeric field is ever used and then cleard
by storing zero to it, the field on disk will be filled with hex 30 which
is the ANSI code for 0. dBase can, therefore, distinguish between null
and empty for a numeric field. dBase cannot distinguish between null and
empty for a date field as it starts out on disk with empty values, not
null values.

A timestamp field is, however, quite a different story. This is stored on
disk as 8 bytes that represent a 64-bit number. It starts out as 8 bytes
of null so dBase would be in a position to determine if it is null or
empty. But then again this thread started out discusing date fields, not
timestamp fields.

Mervyn.









This brings me back to my original contention. A date field, on disk,
starts out filled with spaces, not nulls. dBase is, therefore, unable to
determine if a date field has
Robert Bravery
2008-10-23 07:29:12 UTC
Permalink
Hi,

I am interested to know how you would convert a blank value, stored blank
date, into an 64 bit number and have this number not equal to null.
A blank "Character" value is translated by dbase in a zero or 0 value. THis
is a value and therefore not null. THis then would translate, as you put it,
by dbase into a date of 1900/01/00. Which is neither blank nor null.
THerefore a blank date equating to null is correct. Which I was trying to
explain. The date type of dbase is not a true date data in the database
world. As you put it, dbase then converts its storage into a 64-bit
interger, same used by others as a date data type. Then displays to you the
date.
So my answer still remains, you cannot store a true date datatype, whether
on disk or in memory (converted by dbase as you put it) as 64-bit interger
and expect a null value to return. My answer was not trying to argue how
dbase stores its date on disk, but rather how date data types are stored and
evaluated in an effort to provide a valid answer as to why a blank date
evaluates to null.

Robert
"When dBase reads a date from disk it converts the 8 characters into a
number internally." That internal number could well be your IEEE 64-bit
number. It was not, and still isn't, important to the point I was
making.
---
avast! Antivirus: Outbound message clean.
Virus Database (VPS): 081022-1, 22/10/2008
Tested on: 23/10/2008 09:29:14 AM
avast! - copyright (c) 1988-2008 ALWIL Software.
http://www.avast.com
Mervyn Bick
2008-10-23 09:30:35 UTC
Permalink
Post by Robert Bravery
Hi,
I am interested to know how you would convert a blank value, stored
blank date, into an 64 bit number and have this number not equal to null.
I have no idea what dBase does with blank dates internally. I have no
idea of how to get to the actual memory address being used to store the
date value. I can get at the actual value stored on disk. For an as yet
unused numeric value the field is stored as a group of hex 00 bytes on
disk, the actual number being dependant on the field specification. A
date field in a DBF table is NOT stored on disk as a 64 bit integer even
if dBase uses a 64-bit integer internally. (A timestamp field is stored
as a 64-bit number.) An as yet unused date field is stored on disk as 8
separate bytes each representing one digit in the date in the form
yyyymmdd and these bytes are intially filled with hex 20 which represents
a space in ANSI. The initial bytes are NOT filled with hex 00 as is the
case with numeric fields.
Post by Robert Bravery
A blank "Character" value is translated by dbase in a zero or 0 value.
THis is a value and therefore not null. THis then would translate, as
you put it, by dbase into a date of 1900/01/00. Which is neither blank
nor null. THerefore a blank date equating to null is correct. Which I
was trying to explain. The date type of dbase is not a true date data in
the database world. As you put it, dbase then converts its storage into
a 64-bit interger, same used by others as a date data type. Then
displays to you the date.
1900/01/00 is not a valid date and dBase shouldn't even try to store it to
disk. Unfortunately dBase is a bit sloppy about this and will store
ctod("1900/01/00") as 18991231 on disk.
Post by Robert Bravery
So my answer still remains, you cannot store a true date datatype,
whether on disk or in memory (converted by dbase as you put it) as
64-bit interger and expect a null value to return. My answer was not
trying to argue how dbase stores its date on disk, but rather how date
data types are stored and evaluated in an effort to provide a valid
answer as to why a blank date evaluates to null.
dBase doesn't store values representing a true date datatype to date
fields on disk. It stores a character representation in the format
yyymmdd. When a blank record is created a date field contains the hex
equivalnt of 8 spaces. When dBase reads those 8 spaces it interprets them
as null. How it stores that internally doesn't matter, to me anyway. For
all I know it is stored internally as 8 consecutive hex 00 bytes even
though it was 8 consecutive hex 20 bytes on disk.

If you store a valid date to a date field and later replace it with null
or ctod(" / / ") or ctod("00/00/00") dBase stores 8 spaces to disk.
dBase cannot, therefore, distinguish between a date field that hasn't been
filled with one that has been emptied. It always interprets 8 spaces as a
null date. Rightly or wrongly this is WAD. If a date field that has 8
spaces on disk is copied into a variable, that variable has a value of
null and it plus anything remains null.

Mervyn
unknown
2008-10-23 13:54:57 UTC
Permalink
On Thu, 23 Oct 2008 11:30:35 +0200 Mervyn Bick
Sender: "Mervyn Bick" <invalid.invalid.invalid>
wrote the following in:
Newsgroup: dbase.programming
Post by Mervyn Bick
1900/01/00 is not a valid date and dBase shouldn't even try to store it to
disk. Unfortunately dBase is a bit sloppy about this and will store
ctod("1900/01/00") as 18991231 on disk.
Mervyn,

May I recommend that you and Robert take a look at the OLH on 'Date and time', some of which I quote
below in a different order than in the OLH.

"dBASE Plus supports two types of dates:

A primitive date that is compatible with earlier versions of dBASE

Primitive dates represent the date only, not the time. (They are considered to be the first
millisecond—midnight—of that date.) Literal dates are delimited by curly braces and are evaluated
according to the rules used by the CTOD( ) function. An invalid literal date is always converted to
the next valid one; for example, if the current date format is month/day/year, {02/29/1997} is
considered March 1, 1997. An empty date is valid and is represented by empty braces: { }."

dBase is not sloppy when storing an invalid date, the automatic conversion to a real date is WAD :-)

Note also that a pair of curly braces represents a *valid* empty date.

To check a date field in a table one should according to the OLH use ISBLANK(). ISBLANK( ) returns
true if a specified expression is blank or null.



"A JavaScript-compatible Date object.

A Date object represents a moment in time. It is stored as the number of milliseconds since January
1, 1970 00:00:00 GMT (Greenwich Mean Time). Although GMT and UTC (a compromise between the English
and French acronyms for Universal Coordinated Time) are derived differently, they are considered to
represent the same time."

The date object is more like the date used by Microsoft, but I do not think that a description of
the way a Microsoft date behaves in an Excel spreadsheet, has any bearing on how a date object in
dbase behaves.


Ivar B. Jessen
Mervyn Bick
2008-10-23 16:00:44 UTC
Permalink
Post by Robert Bravery
Mervyn,
May I recommend that you and Robert take a look at the OLH on 'Date and
time', some of which I quote
below in a different order than in the OLH.
Mm. I knew that. <g> The thread has, however, revolved about the
old-fashioned type.
Post by Robert Bravery
dBase is not sloppy when storing an invalid date, the automatic
conversion to a real date is WAD :-)
It may be WAD because that's what the OLH says is WAD but I still think it
was sloppy design in the first place. On the other hand, that's only my
opinion. If the dBase philosophy is "let the programmer validate
everything" then so be it.

Mervyn.

Mervyn Bick
2008-10-23 06:39:18 UTC
Permalink
Post by Robert Bravery
Mervyn,
I believe youre wrong here. If youre talking about the date data type.
But if your talking about a nornaml char data type. well then thats
different. But I believe the thread is about date data type.
If you want to be specific, data as a data type is NOT stored as
character. But as bytes. Infact all data is stored as bytes. As far as I
know you cannot store a blank byte. It has to represent a byte. My
answer was simplistic, but enough to answer the question.
Most places that I read, all say the same, and that is that dates are
stored as byte intergers, and not characters. When last did you store a
Blank Interger. It is either null or a int value.
"Date variables are stored as IEEE 64-bit (8-byte) integers that
represent dates ranging from January 1 of the year 1 through December 31
of the year 9999, and times from 0:00:00 (midnight) through 11:59:59 PM"
It seems that you CANNOT store a blank interger, by nature of the
datatype. By nature there is always something stored, either a null
value, which is in question here, or an interger value. For Date data
type this is a value of 0-2958465. Which gives you a date value of
between 1900/01/00 and 9999/12/31. But strictly speaking 1900/01/00 is
not a valid date. So the actual valid interger value is 1-2958465. -1
gives an error, as does 2958466. It is not spaces or blanks. It is
either a Null or an interger value, infact it is an 8 byte value.
Firstly, the first 6 words of my previous post were "A date is stored on
disk" and the second paragraph started with "When dBase reads a date from
disk it converts the 8 characters into a number internally." That
internal number could well be your IEEE 64-bit number. It was not, and
still isn't, important to the point I was making.

I was technically incorrect when I said data was stored as characters. As
you have pointed out that should have been bytes. Fair enough but for
today's date those 8 bytes would have the hex values of 32 30 30 38 31 30
32 33 which represent the ANSI values for 2 0 0 8 1 0 2 3 and this is true
whether dBase and/or regional settings dictate that the date be displayed
as 10/23/2008 or 23/10/2008 or 23-10-2008 or whatever.

You don't believe me? Fine but go and conduct your own little experiment
and then we can talk again. Create a table with a date field and, for
good measure, a numeric field. Append a blank. Replace the date with
date() and the the numeric field with a value such as 12345.67. Append a
blank Append a blank and replace the date and numeric fields as above.
(This is to establish a pattern so that you can easily see where the
records are and what is in them without having to delve too deeply into
the structure of a .dbf table or having to bother with counting bytes.)
Close the table and open it in a hex editor.

The unused date field will have 8 bytes of hex 20 which is the ANSI code
for "space". The unused numeric field will be filled with hex 00 which is
the ANSI code for null. If a numeric field is ever used and then cleared
by storing zero to it, the field on disk will be filled with hex 30 which
is the ANSI code for 0. dBase can, therefore, distinguish between null
and empty for a numeric field. dBase cannot distinguish between null and
empty for a date field as it starts out on disk with empty values, not
null values.

A timestamp field is, however, quite a different story. This is stored on
disk as 8 bytes that represent a 64-bit number. It starts out as 8 bytes
of null so dBase would be in a position to determine if it is null or
empty. But then again this thread started out discusing date fields, not
timestamp fields.

Mervyn.
Loading...