Sunday, April 8, 2012

Snowflakes, Outriggers, and Bridges -- By Margy Ross ( Kimball Group )

Nice article by Margy, sharing with you all

Students often blur the concepts of snowflakes, outriggers, and bridges. In this Design Tip, we’ll try to reduce the confusion surrounding these embellishments to the standard dimensional model.
When a dimension table is snowflaked, the redundant many-to-one attributes are removed into separate dimension tables. For example, instead of collapsing hierarchical rollups such as brand and category into columns of a product dimension table, the attributes are stored in separate brand and category tables which are then linked to the product table. With snowflakes, the dimension tables are normalized to third normal form. A standard dimensional model often has 10 to 20 denormalized dimension tables surrounding the fact table in a single layer halo; this exact same data might easily be represented by 100 or more linked dimension tables in a snowflake schema.We generally encourage you to handle many-to-one hierarchical relationships in a single dimension table rather than snowflaking. Snowflakes may appear optimal to an experienced OLTP data modeler, but they’re suboptimal for DW/BI query performance. The linked snowflaked tables create complexity and confusion for users directly exposed to the table structures; even if users are buffered from the tables, snowflaking increases complexity for the optimizer which must link hundreds of tables together to resolve queries. Snowflakes also put burden on the ETL system to manage the keys linking the normalized tables which can become grossly complex when the linked hierarchical relationships are subject to change. While snowflaking may save some space by replacing repeated text strings with codes, the savings are negligible, especially in light of the price paid for the extra ETL burden and query complexity.
Outriggers are similar to snowflakes in that they’re used for many-to-one relationships, however they’re more limited. Outriggers are dimension tables joined to other dimension tables, but they’re just one more layer removed from the fact table, rather than being fully normalized snowflakes.Outriggers are most frequently used when one standard dimension table is referenced in another dimension, such as a hire date attribute in the employee dimension table. If the users want to sliceand-dice the hire date by non-standard calendar attributes, such as the fiscal year, then a date dimension table (with unique column labels such as Hire Date Fiscal Year) could serve as an outrigger to the employee dimension table joined on a date key. Like many things in life, outriggers are acceptable in moderation, but they should be viewed as the exception rather than the rule. If outriggers are rampant in your dimensional model, it’s time to return to the drawing board given the potentially negative impact on ease-of-use and query performance.
Bridge tables are used in two more complicated scenarios. The first is where a many-to-many relationship can’t be resolved in the fact table itself (where M:M relationships are normally handled) because a single fact measurement is associated with multiple occurrences of a dimension, such as multiple customers associated with a single bank account balance. Placing a customer dimension key in the fact table would require the unnatural and unreasonable divvying of the balance amongst multiple customers, so a bridge table with dual keys to capture the many-to-many relationship between customers and accounts is used in conjunction with the measurement fact table. Bridge tables are also used to represent a ragged or variable depth hierarchical relationship which cannot be reasonably forced into a simpler fixed depth hierarchy of many-to-one attributes in a dimension table.In these isolated situations, the bridge table comes to the rescue, albeit at a price. Sometimes bridges are used to capture the complete data relationships, but pseudo compromises, such asincluding the primary account holder or top rollup level as dimension attributes, help avoid paying the toll for navigating the bridge on every query.

Thank You
Vishwas Gupta

Saturday, April 7, 2012

List and Crossttab not matching though using same query

Crosstab is more prone to picking data from cache than a list, so sometime when data is refreshed in database while you running report, your list may show the latest data but crosstab keep showing the old data. It becomes difficult to get the right report as even if you run report multple times crosstab would keep showing the old data, in this case you have to either log off and log in or you wait for few minutes.

Bursting Cognos Report to a file system

here I am demonstrating how to set file system in cognos and get brusted there and also run a script so that bursted reports can be renamed.

1) Go to the dispatcher --  ( See the picture below for reference )


2) Click on the dispatcher link to see the content manager service.


3) Go to the settings tab and click on edit on advance setting line -

 

4) Create following  2 variables and provide the path where you want to get the files bursted.




5) Save it and go to cognos configuration to do the following setting.



6) Run the report that you have to burst with the following settings.



7) You will get the output in D:/Report folder, now you want that bursted reports should get proper names or they should get names according to there bursted key.

8) Create a notepad file giving it a name "burstKey.bat" and paste the following contents and save it in D:\Reports folder.

@echo off
Rem  this batch file is launched with a working directory of \bin 

set dest=D:/Reports

echo "BurstKey Script" >> %dest%/batch.txt
echo Batch File Log >> %dest%/batch.txt

rem       Set the values for the PDF file and XML
rem       to environment variables
rem       the reason for this is the scripting language
rem       cannot read variables as parameters
set parameter_pdf=%1%
set parameter_xml=%2%

rem      Logs the variables to a batch log file
echo values set >> %dest%/batch.txt
echo Parameter_pdf: %parameter_pdf% >> %dest%/batch.txt
echo Parameter_xml: %parameter_xml% >> %dest%/batch.txt

rem      Calls the script file
call %dest%/burstKeyRename.vbs

echo Completed >> %dest%/batch.txt


9) create a file "burstKeyRename.vbs" and paste the following contenets in there and save it in D:\Reports folder.


'
' Last updated : Sept 24, 2007
'
Set objFSO = CreateObject("Scripting.FileSystemObject")
Set xmlDoc = CreateObject("MSXML.DOMDocument")
Set WshShell = WScript.CreateObject("WScript.Shell")
Set WshSysEnv = WshShell.Environment("process")

' Values for Environment can be "user", "system", "volitile" and "process
' The value "process" is the only valid one for this type of operation.
Set WshSysEnv = WshShell.Environment("process")

' To check the value for process environment variable
' this can be tested at the command prompt.
' Uncomment the next line and run this from command line
' WScript.Echo WshSysEnv("parameter_xml")

' Initializing Variables
xFile = WshSysEnv("PARAMETER_XML")    'Name of the XML file
sFile = WshSysEnv("PARAMETER_PDF")    'Name of the PDF file

' ***** Start override section for testing outside Cognos environment *****
'
' Set the following if-then condition to "true" for local testing using files that
' have already been burst.  If the if=then condition  is set to "false", then the
' environment variables must be set properly otherwise a  run-time error will occur.
'
If (false) Then       
    'The filename component of the output file. Do not include the extension.    
    tempName = "D:\C83Outputs\wottcattondxp\TestOutputs\911_1190657394280"
    ' Derive the name of the XML descriptor file    
    xFile = tempName & "_desc.xml"
    ' Derive the name of the new output file.
    sFile = tempName & ".pdf"
End If
'
' ***** End override section for testing outside Cognos environment *****

sMsg = ""
sNewFileName = ""

'
' Read the XML descriptor file created by CRN/Cognos 8 to get the value of the element. This
' value will be used as the new filename. If there is no value in the element, a new file will not
' be created.  It is assumed that the value of the element does not contain any of the
' reserved characters the underlying OS uses for filename.
'
xmlDoc.async = False
xmlDoc.Load xFile

If (xmlDoc.parseError.errorCode <> 0) Then
    sMsg = xFile & " : Error parsing XML " + xmlDoc.parseError.reason       
Else
    Set rootXML = xmlDoc.documentElement
    Set oNodeList = rootXML.childNodes
    iNode = 0   
   
    For Each Item in oNodeList
        If (xmlDoc.documentElement.childNodes.Item(iNode).basename = "burstKey") Then
            sNewFileName = xmlDoc.documentElement.childNodes.Item(iNode).text
            If (sNewFileName = "") Then
                sMsg = xFile & " : element in '" & xFile & "' is empty. Unable to copy file."
            Else
                sMsg = xFile & " : Burst key value is '" & sNewFileName & "'"
            End If

            ' This if condition  was added to support the use of CAMID for the burst key
            if (instr(sNewFilename, "CAMID")) Then
                intFirstPosition = instr(sNewFilename, ":")
                intSecondPosition = instr(intFirstPosition, sNewFilename, ")")
                sNewFileName = mid (sNewFilename, intFirstPosition + 1, intSecondPosition - (intFirstPosition + 2))
                sMsg = xFile + " : Value extracted from CAMID is '" & sNewFileName & "'"               
            End If
        End If
        iNode = iNode + 1       
    Next
   
    Set oNodeList = Nothing
    Set rootXML = Nothing
End If

'   
' If there is a new filename, make of copy of it otherwise just log an error. A local log file with a name
' that is derived using the filename given to the output by CRN/Cognos 8 will be created so it is easy
' to determine if the file copied successfully  or not. Since this information will also be written to the
' persistent log file, writing to the local log file can be bypassed if writing it creates too much clutter in the
' location specified in CM.OUTPUTLOCATION.
'
if (sNewFileName <> "") Then
    sNewFileName = getPath(sFile) & sNewFileName & ".pdf"
    writeLocalLogFile sFile, sMsg & vbCrLf & "Copying file named '" & sFile & "' to '" & sNewFileName & "'" & vbCrLf
    objFSO.CopyFile sFile, sNewFileName
Else
    ' Set this variable to an error message so it will be picked up below when writing to the persistent log file.....
    sNewFileName = ""
    writeLocalLogFile sFile, sMsg & vbCrLf
End If

'       
' Update the persistent log file with the result. This log file will have data appended to it in order to keep the history.
'
sMsg = "----- Start of entry -----" & vbCrLf
sMsg = sMsg & "Date : " & date & vbTab & "Time : " & time & vbCrLf
sMsg = sMsg & vbCrLf & "Original Name :" & sFile & vbCrLf
sMsg = sMsg &  "New Name : " & sNewFileName & vbCrLf
sMsg = sMsg &  "----- End of entry -----" & vbCrLf & vbCrLf

sPersistLogFile = getPath(sFile) & "crn_pdf_rename.txt"
writePersistLogFile sPersistLogFile, sMsg
'
' All done.. Release references to the objects used in this app.
'
Set objFSO = Nothing
Set xmlDoc = Nothing
Set WshShell = Nothing
Set WshSysEnv = Nothing


Function getPath(sFileName)
    sPathOnly = ""
    lastSlashPos = InStrRev(sFileName, "\")

    if (lastSlashPos > 0) Then
        getPath = mid (sFileName, 1, lastSlashPos)
    End If
End Function

'
' writeLocalLogFile
'     Create a log file using a name derived from the filename that was generated by
'    CRN/C8 before it was written to the location specified by CM.OUTPUTLOCATION.
'
Sub writeLocalLogFile(sFileName, sData)
    sLogFile = left(sFileName, instr(sFileName,".")-1) +   "_log.txt"
    writeLogFile sLogFile, sData   
End Sub

'
' writePersistLogFile
'     Write a record to the persistent log file
'
Sub writePersistLogFile(sFileName, sData)
    writeLogFile sFileName, sData
End Sub

'
' writeLogFile
'     Generic routine to open a file, append a record to it and close the file
'
Sub writeLogFile(sFileName, sData)
    If (objFSO.FileExists(sFileName)) Then
        Set logFile = objFSO.GetFile(sFileName).OpenAsTextStream(8)       
    Else
        Set logFile = objFSO.CreateTextFile(sFileName)
    End If

    logFile.Write sData
    logFile.Close
   
    Set logFile = Nothing
End Sub

now follow the 6th step again and you will get proper pdf output with name same as burst key.

Vishy

  


Friday, April 6, 2012

Auto Refresh of HTML output

Put this code into an HTML item and place that at the bottom part of the report, HTML output will refresh every 5 mins.

Vesion - Cognos8.4

-----------------------------------
<script type="text/javascript">
// Licensed Material - Property of IBM
// © Copyright IBM Corp. 2003, 2011
var intval;
var fW = (typeof getFormWarpRequest == "function" ? getFormWarpRequest() : document.forms["formWarpRequest"]);
if ( !fW || fW == undefined)
{

    fW = ( formWarpRequest_THIS_ ? formWarpRequest_THIS_ : formWarpRequest_NS_ );

}
var preFix = "";
if (fW.elements["cv.id"])
{

    preFix = fW.elements["cv.id"].value;

}
var nameSpace = "oCV" + preFix;
if(intval!="")
{

    self.clearInterval(intval);
    intval="";

}

self["RunReportInterval"] = self.setInterval( nameSpace + ".getRV().RunReport()",'600000' );
intval = self["RunReportInterval"];
</script> 

Thursday, April 5, 2012

Dynamic datasource selection - Report Studio Report
















Settings at the Framework Manager


See the prompt and prompt static value which shows the data source that you have for selection. Name of the parameter should be same as given at FM level.


Thanks
Vishwas

#prompt# macro

We use #prompt(...)# macro function in the FM so that few things can be dynamically decided. But #prompt # works with one value only I mean you get a text box prompt when you run a report using #prompt # macro.
but if you want that user should be able to see multiple values so that he can select one of values rather then typing in the text box prompt.
It will also help developer as there is no chance user making mistake by typing wrong value.
To do this just create any prompt at the report studio level with the same name that you have given to prompt in the #prompt(..)#.
Report studio prompt will override the FM prompt and values you select will be taken as #macro # prompt value only. This way you will be able to do that.

Thanks
Vishwas

Wednesday, April 4, 2012

Cognos cube fast build


while building cube two things that can make cube creation fast is

i) making datasource which is bringing fact ..UNCHECK GENERATE CATEGORIES.Only dimensionsal source  should populate

categories and not the fact source.

ii) If last level in dimensions which is defined unique using the surrogate key then make sure that you remove -  

VERIFY UNIQUNESS selection for that dimension.

Report failing after cube rebuild


while building transformer model we always define the unique id and label but most of the time we forget to define

the category id's which should not be left for transformer to decide while building the cube. Category id should

come from the dimension( database) itself. It will help when cube is rebuild due to any xyz issue as all your

exisiting cube based report will keep on working otherwise your exisitng report will start throwing the error as it

will search for cognos defined category id's in the cube which is not there anymore due to rebuild of the cube.

So always define three minimum things for a level in a dimension id,lable and category id so that life is easy.

IQD not getting created via Cognos FM

if for query subject corressponding  iqds are not getting created then make sure that in package definition ..SELECT COMPONENTS AND CHILDRENs is selected for namespace in which iqd query subjects are located.

short cuts v/s alias

Short cuts -- should be used when you need less maintainance as well as when you have star schema grouping and you

need one of the query subject to behave as confirmed dimension.

Alias -- mainly used when you want a role playing dimension.

Joins at which layer ??


1) Where to define joins in FM layers, few says at database layer and few says at business layer.
   Let us see where to do what -

a)  data layer - making join at the data base layer makes more sense when you are modelling a relational data so

that you only get those tables in the query which you really are using. This is called minimized query.

b) business layer -  if you define joins here or I say if you don't define joins in the data layer then your data

layer is free, on other layers you can redifne joins when need arise.

Saturday, March 31, 2012

Time Dimension Design from Cognos Perspective

Cognos do provide tecgniques like determinants at FM level to avoid double counting, but we cognos guys should make sure we don't reach to that level if possible.

The best is to flatten the hierarchy, most of the time we face this issue in time dimension but if we can flatten by not having surrogate key for month level and only have it for day level, we will be good. Directly join month level fact with this day level surrogate key...


Cognos FM best practices

We all know that implementing best practices at FM  is very important as all reports/cubes/metric designer are dependent on it.

We may not follow best practice at report studio or analysis studio level as these things don't have any other objects which are dependent on them.

But at cube and at FM and at metric designer it is imperative to follow best practice.

Let us discuss PRACTICALY best practice at FM level.

Create following layers

Data layer - vanilla database table objects , no joins at this layer. Define the query items property here so that following layers can inherit that.

Business layer - crate joins and determinants and give business names to query items.


Dimension leyer - if you are doing a DMR.

Presentation layer or short cut layer -  better you call it a short cut layer so that you know you have to use short cut here , you might want crate star schema groupings here if required.

If there is a confirmed dimension that lies in 2 star schema grouping better you make join again at presentation layer b/w short cuts so that cognos can do stitching .






Thank You,
Vishwas Gupta

Stitched query, star schema grouping and cognos FM

Many a times instead of using cognos own automatic functionalities , seen people using star schema groupings.

Start schema groupings are mainly used when you want to present clean picture of query subjects to the user so that they don't mix business subjects in the report.

It should not be used when you are having multiple facts in design as multiple facts will be taken care automatically when cardenilities are set 1-n between facts and dimensions. FM will automatically resolve multiple paths via stitched queries.



Thank You,
Vishwas Gupta

Determinants and FM layers

Determinants in framework manager has to be defined in the layer where you have joins defined.

If you have defined joins at business layer and defined determinants at dimension layer then nothing will work !!



Thank You,
Vishwas Gupta