Archive for category Microsoft SQL Server

Date formatting in SQL Server

One of the comments that often comes up in the Introduction to SQL course, is that Oracle provides much more in the way of formatting dates than SQL Server. I’m aware that many people are of the view, “formatting should be done in the client application, it’s not the job of the database”. I agree with that to a certain extent – good practice will often dictate that formatting should be done outside of the database – but there are times when I get frustrated at the limitations of SQL Server in this area.

For one thing, SQL Server provides the ability to send emails from within the database. I would have thought that this alone justifies better formatting than is currently available.

Secondly, we don’t live in a perfect world where we have the choice to put all code in the correct layer every time.  We might only have control of the data tier, with a presentation tier displaying everything returned from a stored procedure, and we may not be able to change the code in that presentation tier.

And finally, if date formatting should never be done in the database, why bother providing the CONVERT function at all? To provide it, but with only a very limited set of formats, seems inconsistent.

So one of the first thing I did when I came to SQL Server from Oracle was to code up a quick and simple date formatting routine, that I wanted to work in a similar way to Oracle’s TO_CHAR function for dates. I only included the most basic format support and expected to have to extend it regularly as my requirements got more complex, but in fact I don’t think I’ve done much in the way if altering it, it pretty much has met my requirements whenever I’ve needed it. I still use CONVERT when one of the Microsoft supplied date formats is acceptable, but this allows me a bit more flexibility. Being a user defined function, performance will be affected, so I use it sparingly, but it definitely comes in handy. It’s designed to accept a date, and then an Oracle-style format, like ‘DD MON YY HH:MI’, and it returns the input date in the specified format, as a string. Alternatively I allow the passing of special strings like ‘Standard Date’, ‘Time only’ etc, which can be customised for the requirements of a particular project.

As it was almost the first UDF I wrote, it doesn’t use best coding practices, and I can now see several improvements I could make, but for the few times I need it, this works fine for me. If it’s useful for you, feel free to copy it, and maybe make some of those long awaited improvements I never got round to making.

Dean

=====================================================================================
-- Author: D Bullen
-- Description: This procedure accepts a date value, formats it, and returns it as a string.
-- The format is passed in as the @Format parameter, and it accepts the following:
-- DD - Date
-- MM - Month number
-- MNTH - Month Name
-- MON - Month Short Name
-- YYYY - 4 digit year
-- YY - 2 digit year
-- HH - Hours
-- MI - Minutes
--
-- Alternatively prefined standard formats can be passed in and translated into basic
-- format types, e.g. 'Standard Date & Time' - translated to 'DD MNTH YYYY at HH:MI'
-- =====================================================================================
CREATE FUNCTION [General].[FormattedDate]
( @DateIn DATETIME
,@Format VARCHAR(100))
RETURNS VARCHAR(100)
AS
BEGIN
DECLARE @FormattedDate VARCHAR(100)

IF @Format = 'Standard Date' SET @Format = 'DD MNTH YYYY'
IF @Format = 'Standard Date & Time' SET @Format = 'DD MNTH YYYY at HH:MI'
IF @Format = 'Short Date' SET @Format = 'DD MON YY'
IF @Format = 'Short Date & Time' SET @Format = 'DD MON YY HH:MI'
IF @Format = 'Time Only' SET @Format = 'HH:MI'

DECLARE @DateNum VARCHAR(2)
SET @DateNum = CASE WHEN LEN(CAST(DATEPART(day, @DateIn) AS VARCHAR)) = 2 THEN CAST(DATEPART(day, @DateIn) AS VARCHAR) ELSE '0' + CAST(DATEPART(day, @DateIn) AS VARCHAR) END
DECLARE @MonthNum VARCHAR(2)
SET @MonthNum = CASE WHEN LEN(CAST(DATEPART(month, @DateIn) AS VARCHAR)) = 2 THEN CAST(DATEPART(month, @DateIn) AS VARCHAR) ELSE '0' + CAST(DATEPART(month, @DateIn) AS VARCHAR) END
DECLARE @MonthName VARCHAR(20)
SET @MonthName = DATENAME(month, @DateIn)
DECLARE @MonthShortName VARCHAR(3)
SET @MonthShortName = SUBSTRING(@MonthName, 1, 3)
DECLARE @YearName VARCHAR(4)
SET @YearName = DATENAME(year, @DateIn)
DECLARE @YearShortName VARCHAR(2)
SET @YearShortName = SUBSTRING(@YearName, 3, 2)
DECLARE @Hours VARCHAR(2)
SET @Hours = CASE WHEN LEN(CAST(DATEPART(hour, @DateIn) AS VARCHAR)) = 2 THEN CAST(DATEPART(hour, @DateIn) AS VARCHAR) ELSE '0' + CAST(DATEPART(hour, @DateIn) AS VARCHAR) END
DECLARE @Minutes VARCHAR(2)
SET @Minutes = CASE WHEN LEN(CAST(DATEPART(minute, @DateIn) AS VARCHAR)) = 2 THEN CAST(DATEPART(minute, @DateIn) AS VARCHAR) ELSE '0' + CAST(DATEPART(minute, @DateIn) AS VARCHAR) END

SET @FormattedDate = @Format
SET @FormattedDate = REPLACE(@FormattedDate, 'DD', @DateNum)
SET @FormattedDate = REPLACE(@FormattedDate, 'MM', @MonthNum)
SET @FormattedDate = REPLACE(@FormattedDate, 'MNTH', @MonthName)
SET @FormattedDate = REPLACE(@FormattedDate, 'MON', @MonthShortName)
SET @FormattedDate = REPLACE(@FormattedDate, 'YYYY', @YearName)
SET @FormattedDate = REPLACE(@FormattedDate, 'YY', @YearShortName)
SET @FormattedDate = REPLACE(@FormattedDate, 'HH', @Hours)
SET @FormattedDate = REPLACE(@FormattedDate, 'MI', @Minutes)

IF @Format = 'Standard Date & Time' AND @FormattedDate LIKE '%at 00:00' SET @FormattedDate = REPLACE(@FormattedDate, 'at 00:00', '')

RETURN @FormattedDate

END

About the poster
Dean Bullen
Dean Bullen is Impartica’s SQL course leader and has been working with SQL since 1998. He previously worked for Oracle and has been working with SQL Server since 2006.

Joining tables on values in different records

At a recent running of our Introduction to SQL course, a delegate asked me a great question, when we were on the subject of joining multiple tables. He had a requirement to link two tables, based on values that were not held in different fields of the same record, but in the same field in different records. The platform was SQL Server 2005.

Problem

Specifically, he had a table which held details of stocks traded. For simplicity, let’s just image it has two columns – amount_sold and date_sold. Such a table might look like this:

SELECT amount_sold, date_sold FROM stock_sales 

amount_sold  date_sold
-----------  -----------------------
1700         2010-02-15 00:59:49.190
1900         2010-02-16 05:47:49.203
1400         2010-02-24 20:11:45.873

The table he needed to join to held exchange rate values at different points in time. Again stripped down to the columns relevant to our problem, we could imagine it would look something like this:

SELECT rate, effective_date FROM exchange_rate 

rate      effective_date
--------- -----------------------
1.09      2010-01-01 00:00:00.000
1.12      2010-02-17 11:14:00.000
1.2       2010-02-20 13:24:00.000

What he wanted to do – as you can probably already anticipate – is to find the appropriate exchange rate for each sale, in other words to join the stock_sales table to exchange_rate, where date_sold is greater than or equal to effective_date, but less than effective_date in the next exchange_rate record.

Great problem! In 12 years I find it difficult to believe I haven’t come across this type of problem before, but if I have I can’t remember it! So it gave me some thinking to do.

Here is my answer to the problem. Please let me know if you can find a better one!

My solution

My immediate thought was that if the exchange_rate table was in a format that held “effective from” and “effective to” columns, then the solution would be easy to achieve with a standard non-equijoin. So if it looked like this:

rate    effective_date_from     effective_date_to
------- ----------------------- -----------------------
1.09    2010-01-01 00:00:00.000 2010-02-17 11:14:00.000
1.12    2010-02-17 11:14:00.000 2010-02-20 13:24:00.000
1.2     2010-02-20 13:24:00.000 2010-02-27 14:44:49.830

Then this query would give him the data he needed.

SELECT amount_sold, date_sold, rate
FROM stock_sales
INNER JOIN exchange_rate
ON (stock_sales.date_sold >= exchange_rate.effective_date_from AND stock_sales.date_sold < exchange_rate.effective_date_to)

amount_sold date_sold               rate
----------- ----------------------- ------
1700        2010-02-15 00:59:49.190 1.09
1900        2010-02-16 05:47:49.203 1.09
1400        2010-02-24 20:11:45.873 1.2

So, if I could get my exchange_rate data into that format, then I could use an inline view to get my desired result. (I could also use a common table expression, and this would be slightly easier, but as these weren’t covered on the course I was delivering I preferred to stick to inline views). So how to get my data in the format I wanted?

I thought a good starting point would be to generate two copies of the exchange_rate table data. If I could then give each one an ID column, then I could join them together, matching each row in the first table to the following row in the second table. Then the effective_date column from the first table would be my “effective_date_to” column, and the same column from the second table would be “effective_date_from”. To generate the “ID” column for each, I used the ROW_NUMBER() function.

The query

SELECT ROW_NUMBER() OVER (ORDER BY effective_date) rownum, rate, effective_date FROM exchange_rate

gives me my first copy of the data, with a numeric “key” against each record, like so:

rownum   rate    effective_date
-------- ------- -----------------------
1        1.09    2010-01-01 00:00:00.000
2        1.12    2010-02-17 11:14:00.000
3        1.2     2010-02-20 13:24:00.000

I can therefore slightly adapt that, subtracting 1 from the generated number, to provide a second copy of the data:

SELECT (ROW_NUMBER() OVER (ORDER BY effective_date))-1 rownum, rate, effective_date FROM exchange_rate 

rownum   rate    effective_date
-------- ------- -----------------------
0        1.09    2010-01-01 00:00:00.000
1        1.12    2010-02-17 11:14:00.000
2        1.2     2010-02-20 13:24:00.000

Each of these queries can therefore be made into inline views, and these can then be joined on the rownum column. This will join together records in the first result set with the next record on in the second. (If I was happy to use a common table expression I wouldn’t need to take this second step, I could have declared a CTE from the first query and used it twice.)

SELECT exchange_rate1.rate
,      exchange_rate1.effective_date AS effective_date_from
,      exchange_rate2.effective_date AS effective_date_to
FROM
  (  SELECT ROW_NUMBER() OVER (ORDER BY effective_date) rownum, rate, effective_date
     FROM exchange_rate) exchange_rate1
LEFT OUTER JOIN (SELECT (ROW_NUMBER() OVER (ORDER BY effective_date))-1 rownum, rate, effective_date
                 FROM   exchange_rate ) exchange_rate2
ON (exchange_rate1.rownum = exchange_rate2.rownum)

rate   effective_date_from     effective_date_to
------ ----------------------- -----------------------
1.09   2010-01-01 00:00:00.000 2010-02-17 11:14:00.000
1.12   2010-02-17 11:14:00.000 2010-02-20 13:24:00.000
1.2    2010-02-20 13:24:00.000 NULL

This is now pretty close to what we want. I just need to handle trades made since the last exchange_rate became effective – i.e the NULL that appears at the end of my result set. There are a couple of ways to do it, but for simplicity I will just use ISNULL to replace the null with any future date. Any future date is fine, as no future trades are likely to be recorded in my stock_sales table.

SELECT exchange_rate1.rate
,      exchange_rate1.effective_date AS effective_date_from
,      ISNULL(exchange_rate2.effective_date, DATEADD(day, 1, getDate())) effective_date_to
FROM
  (  SELECT ROW_NUMBER() OVER (ORDER BY effective_date) rownum, rate, effective_date
     FROM exchange_rate) exchange_rate1
LEFT OUTER JOIN (SELECT (ROW_NUMBER() OVER (ORDER BY effective_date))-1 rownum, rate, DATEADD(millisecond, -1, effective_date) effective_date
                 FROM   exchange_rate ) exchange_rate2
ON (exchange_rate1.rownum = exchange_rate2.rownum)

rate   effective_date_from     effective_date_to
------ ----------------------- -----------------------
1.09   2010-01-01 00:00:00.000 2010-02-17 11:14:00.000
1.12   2010-02-17 11:14:00.000 2010-02-20 13:24:00.000
1.2    2010-02-20 13:24:00.000 2010-02-26 15:38:29.493

Having got my data in to the format I was looking for, I can now join in my stock_sales table using a standard non-equijoin:

SELECT amount_sold, date_sold, rate FROM stock_sales
INNER JOIN
	(SELECT exchange_rate1.rate
	 ,      exchange_rate1.effective_date AS effective_date_from
	 ,      ISNULL(exchange_rate2.effective_date, DATEADD(minute, 1, getDate())) effective_date_to
	 FROM
	   (  SELECT ROW_NUMBER() OVER (ORDER BY effective_date) rownum, rate, effective_date
 		 FROM exchange_rate) exchange_rate1
 	 LEFT OUTER JOIN (SELECT (ROW_NUMBER() OVER (ORDER BY effective_date))-1 rownum, rate, DATEADD(millisecond, -1, effective_date) effective_date
					 FROM   exchange_rate ) exchange_rate2
	 ON (exchange_rate1.rownum = exchange_rate2.rownum)
	 ) exchange_rates_range
ON (stock_sales.date_sold >= exchange_rates_range.effective_date_from AND stock_sales.date_sold < exchange_rates_range.effective_date_to)

amount_sold date_sold               rate
----------- ----------------------- ----------------------
1700        2010-02-15 00:59:49.190 1.09
1900        2010-02-16 05:47:49.203 1.09
1400        2010-02-24 20:11:45.873 1.2

Conclusion

This was a nice little problem, and one which I’m sure will have many practical uses. I would be interested in hearing anyone else’s alternative ways to solve it.

 

About the poster

Dean Bullen

Dean Bullen is Impartica’s SQL course leader and has been working with SQL since 1998. He previously worked for Oracle and has been working with SQL Server since 2006.

SQL Server 2008 row constructors

In today’s blog I’m going to look at a new feature in SQL Server 2008 which, while it doesn’t provide any functionality not previously possible, it certainly makes a couple of jobs much easier. Some people are calling this feature row constructors. It has basically two uses.

Firstly, you can insert more than one row into a table using a single INSERT statement. Here is the syntax:

INSERT INTO monthly_sales (office_id, sales_year, sales_month, sales_value)
VALUES (42, '2009', '01', 324877)
, (42, '2009', '02', 329483)
, (42, '2009', '03', 381002)
, (89, '2009', '01', 50228)
, (89, '2009', '02', 49763)
, (89, '2009', '03', 49883)

Quite a neat addition, certainly very readable. Of more practical use possibly, the six rows to be inserted now either stand or fall together: if any of the value sets contain invalid data, the whole statement will fail and no rows will be inserted. In situations where this behaviour is desirable, this single statement can replace a potentially complex T-SQL block. (But of course beware that in some cases, you may NOT want each row to rely on others, and this could end up being a curse rather than a blessing).

The second use for row constructors is in formulating an on-the-fly virtual table, of the type you may previously have used a SELECT/UNION combination for.

Consider the following requirement. Using values from the table above, show the monthly sales for office 42 for every month in 2009. Since we only have data for the first three months, we will have to join the table to one we create on the fly. The following statement would probably be the best fit in previous versions of SQL Server:

SELECT ilv.month_name, ISNULL(ms.sales_value, 0) AS sales_value
FROM monthly_sales AS ms
RIGHT OUTER JOIN
( SELECT '01' AS month_name
UNION
SELECT '02' AS month_name
UNION
SELECT '03' AS month_name
UNION
SELECT '04' AS month_name
UNION
SELECT '05' AS month_name
UNION
SELECT '06' AS month_name
UNION
SELECT '07' AS month_name
UNION
SELECT '08' AS month_name
UNION
SELECT '09' AS month_name
UNION
SELECT '10' AS month_name
UNION
SELECT '11' AS month_name
UNION
SELECT '12' AS month_name) AS ilv ON (ilv.month_name = ms.sales_month AND ms.office_id = 42 AND ms.sales_year = '2009')
ORDER BY 1

Results:

01 324877
02 329483
03 381002
04 0
05 0
06 0
07 0
08 0
09 0
10 0
11 0
12 0

However the row constructor allows us to use a neater syntax to build up the inline view. Try this:

SELECT * FROM (VALUES ('01'),('02'),('03'),('04'),('05'),('06'),('07'),('08'),('09'),('10'),('11'),('12')) AS ilv (month_name)

You will see it gives us the same result as the sequence of SELECTs and UNIONs in the main query, i.e. a single column, 12 row table, called ilv, with the column name of month_name. We can then join this into our monthly_sales table as follows:

SELECT ilv.month_name, ISNULL(ms.sales_value, 0) AS sales_value
FROM monthly_sales AS ms
RIGHT OUTER JOIN
( VALUES ('01'),('02'),('03'),('04'),('05'),('06'),('07'),('08'),('09'),('10'),('11'),('12')) AS ilv (month_name)
ON (ilv.month_name = ms.sales_month AND ms.office_id = 42 AND ms.sales_year = '2009')
ORDER BY 1

Results:

01 324877
02 329483
03 381002
04 0
05 0
06 0
07 0
08 0
09 0
10 0
11 0
12 0

You are not restricted to one column, that’s just the example I chose to use.

Although it probably won’t make a major difference to your SQL life, keep row constructors in mind when working on SQL Server 2008.

About the poster

Dean Bullen

Dean Bullen is Impartica’s SQL course leader and has been working with SQL since 1998. He previously worked for Oracle and has been working with SQL Server since 2006.